Skip to content

Two-legged identifier-liveness gate over the published skills catalog (#13678) - #13892

Merged
huangyiirene merged 2 commits into
mainfrom
claude/issue-13678-skill-identifier-liveness-gate
Aug 31, 2026
Merged

Two-legged identifier-liveness gate over the published skills catalog (#13678)#13892
huangyiirene merged 2 commits into
mainfrom
claude/issue-13678-skill-identifier-liveness-gate

Conversation

@huangyiirene

Copy link
Copy Markdown
Collaborator

Fixes#13678

A two-legged identifier-liveness gate over the published skills/** catalog, plus its self-test and its registrations with the farm's convention gates. Zero skills/** edits — the gate reads that corpus, it does not fix it.

The two predicates, stated precisely

Leg 1 — phantom row. A CITATION is a markdown table row, outside a fenced block, whose FIRST cell is exactly one backticked token of identifier shape (dotted segments, optional [] as in bands[].key), optionally followed by a short parenthetical annotation. It is LIVE when every dotted segment appears as a word token in the IMPLEMENTATION INDEX. Zero hits on any segment is a red naming file, line, identifier and the failing segment.

The index is built from packages/** source only — never from skills/** and never from content/docs/**. That exclusion is the load-bearing line in the leg: an index containing the teaching corpus would find every citation in the corpus that cites it, so every row would be live, the finding set would be empty on any tree, and the gate would be a green light wired to nothing. dist/ is skipped for the mirror-image reason — a stale build output keeps a retired identifier alive for as long as nobody cleans it, making the verdict a function of the developer's working directory. Both are --self-test assertions, not prose.

Three deliberate refusals in the extractor, each measured rather than assumed:

  • The trailing annotation.`none` (default) is a citation of none. A first cut required the cell to be exactly a backticked token; that omission silently dropped every annotated row, and on the Leg 2 side it manufactured two false reds by making complete tables read as 4-of-5 and 3-of-4.
  • The negative column. A migration table's left column cites the dead spelling BY DESIGN (| Legacy | CEL |, | Old | New |). Reddening those runs the gate backwards. Columns whose HEADER names the pre-migration or negative side are structurally out of the population — structurally, not by ledger entry, so those files never enter a ledger whose budget a later edit could spend.
  • The fence. Code fences are skipped. This is the opposite choice from check-corpus-claim-drift, deliberately: that gate exists because its defect lived in a comment inside a fence. This one reads table rows, and a pipe inside a fenced example is not one.

Leg 2 — missing row. A BINDING is a human assertion, in a table in the gate, that one named section of one file is exhaustive over one named schema symbol. For each binding: resolve the symbol from its declared source (z.enum or a named string-literal union, comments stripped first), locate the heading, take the section down to the next heading of the same-or-higher level, and require every member to appear inside some backticked span in that section. A missing member is a red.

Two structural safety rules: the anchor must be unique in its file (an ambiguous heading fails rather than silently first-matching a different table than the row's author read), and the symbol must resolve (a binding naming a deleted enum is STALE and fails — without that, deleting the schema would turn the row green, the exact inversion of what the row is for).

Scope is the SECTION, not the file: an exhaustiveness claim covers what a reader sees under that heading, and a file-wide pool credits a member mentioned 800 lines away. Measured, the difference is real — NavItemVariant reads 7-of-9 against its section and 8-of-9 against its whole file.

Why Leg 2 is registered rather than inferred — the false-red measurements

The scope note asks for a conservative "presented as exhaustive" predicate. Three candidates were built and measured on the real corpus before this shape was chosen.

PredicateRecallPrecisionVerdict
(a) auto-bind to any enum the table is a subset of17 bindings1 true positive in 3cannot fail CI
(b) keyword exhaustiveness markers ("all", "every", "the N types")18 of 155 tablespoor before member comparisonunreliable claim detector
(c) table names its own schema symbol in backticks1 of 155 tables, and that one hedgedhighno recall, and zero skills/** edits permitted

(a) in detail. 17 bindings, 9 candidate reds on the first cut. Six were artefacts: two from the trailing-annotation bug above (MetadataLockSchema read 3-of-4 and SeedMode 4-of-5 while both tables were complete), one from an enumeration split across several tables in one file (FlowNodeAction, whose own lead says "20 built-in node types"), and three from prose naming its own partiality. Of the three that survived every fix, two are one enumAggregationMetricType carries six aggregation functions plus three number/string/boolean members whose comments read "Custom SQL expression returning X". A table titled "Basic Aggregation Functions" listing exactly the six is CORRECT, and this predicate calls it false. In two files.

(b) in detail. The top-ranked hit is skills/objectstack-ai/SKILL.md "Three action types dispatch headlessly" — it matches on both "every" and a numeral and is a deliberate 3-of-6 subset. Most other matches belong to a neighbouring sentence, not the table ("every author must remember to re-attach", "all optional").

So the table-driven shape is a measurement result, not a preference, and it follows the idiom PR #13742 established next door: the engine takes the table as a parameter and --self-test drives synthetic rows through it, so genericity is proven without filling the table. The heuristic is not discarded — it ships as --suggest, a non-failing discovery mode that lists unregistered candidates, so the registry has a feeder and the numbers above stay reproducible instead of living only in a comment.

False-red analysis on the current corpus — every hit dispositioned

Leg 1: 500 citations over 48 published files, checked against 91,177 word tokens from 5,405 files. Five rows survive the predicate. None is a phantom, and they are two structural classes:

SiteIdentifierKindWhy it cannot be indexed here
objectstack-pm-dispatch/SKILL.md:98,102,103backlogRepo, conventionsFile, routingLabelPrefixskill-owned-configKeys of a config file the CUSTOMER writes, consumed by the skill's own prose. The skill IS the implementation; there is no repo symbol to grep.
objectstack-ui/SKILL.md:557,562dayStart, showMidnightsibling-repoLive in ../objectui at packages/plugin-gantt/src/GanttView.tsx. This repo ships backend only, packages/console/dist is gitignored, and sdui.manifest.json does not carry gantt plugin props — checked, not assumed.

A sixth, bare_field in objectstack-formula, sits under a | Legacy | CEL | header and is removed structurally by the negative-column rule rather than ledgered.

Leg 2: 9 registered bindings. Eight are complete today and are now pinned against future drift: ApiMethod, ObjectAccessScopeSchema, HookEvent (x2 files), LifecycleClassSchema, SeedMode, ReportType, ActionType. The ninth is a real, known gap, ledgered rather than fixed: ### Navigation Item Types documents 7 of the 9 NavItemVariant members — action and component have no row. Filed as #13890, a sub-issue of the sweep program.

Registering ActionType against the UI skill's complete table while the AI skill documents a deliberate 3-of-6 subset of the same enum is the registry earning its keep: a binding names a SECTION, so the correct table is pinned and the subset is left alone.

One finding worth a reviewer's eye, which the gate deliberately does not act on.ApproverType has ten members including role, and the doc documents nine. Documenting the tenth would trip check:role-word, which makes "role" a reserved-forbidden word across this same corpus. Two gates over one corpus can be put in direct conflict by a schema member's name. No binding is registered for it, and that is a decision rather than an oversight.

Robustness to the pending sweep merges

The brief asks that in-flight sweeps not redden this. Checked, not assumed: #13833 touches objectstack-i18n, #13861 touches objectstack-upgrade and adds skills/spec-property-retirement. Neither touches any file this gate ledgers. Predicates are identifier-level throughout — there is no count pin anywhere in either leg, so a sweep that adds or rewrites prose moves nothing unless it adds a citation that greps to zero, which is the gate working.

The two ledgers move in OPPOSITE directions, deliberately

  • Leg 1 exemptions — PRUNE-ONLY.kind and note are human judgements about why an identifier is unreachable from this repo; nothing in the tree encodes them, so an --update that rewrote this list would invent classifications it cannot know. --update only DELETES entries the scan no longer reaches. Shrink-only by construction, not by policy: there is no code path that grows it.
  • Leg 2 gaps — REWRITTEN FROM THE TREE, shrink-only in spirit. A gap is a measured count and is fully derivable, so --update moves it whichever way the tree moved. Only policy tells those apart, so the new-gap and grown-gap messages mark that path ⛔ MAINTAINER-ONLY while the author's own remedy is offered first and unmarked.

A gap that SHRANK fails and asks for --update — an improvement that does not move the ledger leaves budget behind for a later edit to spend silently.

Ablation — the gate CAN fail, on both legs

Commit-before-mutate: the implementation was committed first, so the restore leg points at HEAD and HEAD really holds it. Each leg proves its mutation reached DISK before any verdict is read (an editor's exit code proves nothing — sed/perl -i/replace all exit 0 on zero hits), and each restore is proven by blob-hash equality against the HEAD blob, never by an exit code. The script carries a trap ... EXIT INT TERM, restores with git checkout HEAD -- PATH (never the bare form, which restores from the index and would hand the mutation straight back at exit 0), and treats an empty hash as FAILURE rather than "nothing to compare".

=== BASELINE (unmutated tree) ===
exit=0
restored OK skills/objectstack-ui/SKILL.md blob=53121751928a… == HEAD:…
restored OK skills/objectstack-api/SKILL.md blob=c73e66997112… == HEAD:…
=== LEG 1 ABLATION: inject a PHANTOM ROW ===
injected-token count: 1 blob 53121751928a… -> c134f11ea385…
GATE exit=1
[leg1-phantom] skills/objectstack-ui/SKILL.md:642 cites `zzPhantomKeyNeverImplemented`
and the segment `zzPhantomKeyNeverImplemented` has ZERO hits in packages.
restored OK blob=53121751928a… == HEAD gate after restore exit=0
=== LEG 2 ABLATION: delete a documented MEMBER row ===
removed-row count now: 0 blob c73e66997112… -> 11673302ee99…
GATE exit=1
[leg2-missing-row] skills/objectstack-api/SKILL.md — ## API Methods (Operations) is
registered exhaustive over `ApiMethod` (packages/spec/src/data/object.zod.ts) and
does not document `bulk`.
restored OK blob=c73e66997112… == HEAD gate after restore exit=0

git status --porcelain is empty on the tree this PR was pushed from.

Registrations

  • check-ratchet-remedy-authority — a CONTROL row. The sweep classified this gate markedbefore the row was written, and the row is recorded from that verdict rather than from author intent; the corpus is set-equality audited both ways, so the row was demanded, not volunteered. Getting there needed the remedy text to carry real shrink testimony — the first wording said "weakens the gate", which the detector cannot anchor. Farm now: 12 marked, 6 refusing, 27 control rows.
  • bare-root-worklistREFUSE-WIDE for IMPL_ROOTS packages. The index reads 5,405 of 5,640 tracked files under that root (96%), so the population is not a subset of the root, it IS the root; a true declaration would name this gate on every card touching any package. Its other root, skills, IS declared beside the constant via ROOT_DIR_WATCH_HINTS — small, and the surface the gate exists for.
  • check-watch-hint-literal — the watch-hint self-test asserts the spelling without writing a second declaration site. Spelling the whole const NAME = [...] as a regex literal creates one, and the gate then reported "2 declaration sites — this gate cannot judge a declaration it cannot locate". Measured, then fixed.
  • lint.yml — one step in the job that carries the whole check:* family, with no paths filter. That matters more here than for its neighbours: the gate's two inputs are skills/** and packages/**, so a filter on either would let the OTHER side drift a member in or an identifier out without ever running the check that notices.

Verification

All readings taken on 1d84cf3b4, a clean tree, exit codes captured before any pipe.

node scripts/check-skill-identifier-liveness.mjs --self-test exit=0
node scripts/check-skill-identifier-liveness.mjs exit=0
Leg 1: 500 citations / 48 published files / 91,177 tokens / 5 exemptions
Leg 2: 9 registered exhaustive sections, 1 ledgered gap

Full derived family union from node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (33 families — all run, both invocation spellings), every one exit 0: entry-guard · parse-guard · watch-hint-literal · ratchet-remedy-authority · skill-identifier-liveness · agent-test-spelling · cli-command-ids · bash32-floor · pnpm-filter-targets · node-version · pnpm-acquisition · nul-bytes · cross-package-test-inputs · required-contexts · shard-attestation · stall-guard-budget · turbo-task-graph · workflow-status-functions · type-check-coverage · type-check-debt · dual-build-cjs-loads · pm-dispatch-gates · bare-root-worklist --self-test · aggregator-roster · ci-filter-parity · position-name-fold-loaders · self-test-wired · self-test-workflow-commands · step-collectors · whole-set-label-write · ci-failure --self-test · docs-audit/check-drift-comment.

Both convention-triggered obligations for adding a gate script were run and are green. check:type-check-debt was run after building the closure (turbo run build, 70/70 successful), because its --re-measure half silently measures a different world without it: OK — 29 ledger entries re-measured, 1531 raw tsc errors, none above its recorded number.

Repo-wide pnpm lint (eslint . --no-inline-config) run in full: exit 0 in 59s. No narrowing claimed.

node scripts/check-test-completeness.mjs returns exit 3 = PREREQUISITE NOT MET (it grades a saved turbo run test log that only CI produces). Recorded as NOT MEASURED — it is not a finding and not a failure.

The two edited gate scripts (check-ratchet-remedy-authority.mjs, bare-root-worklist.mjs) carry no vitest suite — repo-wide, no test file names either — so their --self-test IS their suite, and both were run green.

Changeset

None, and skip-changeset applied. Verified rather than assumed: the diff is scripts/**, .github/workflows/lint.yml, and one alias line in the private root manifest (@objectstack/spec-monorepo, private: true). Nothing under packages/, apps/ or examples/ changed, so this releases nothing — the workflow's own words for this case are "this PR edits a CI-internal script is the textbook skip-changeset case".

Scope

scripts/ and .github/workflows/lint.yml are not governed surfaces — regular code lane, opened as draft for the PM. Adjacent cards #13859 and #13835 are deliberately untouched. #13859 is adjacent but not subsumed: it is about a retirement-pin scanning one markdown file, a different instrument over a different question — this gate would not have caught it, because a withdrawn claim's sentence is semantics, and Leg 1 catches existence only.


Generated by Claude Code

…ills catalog
Leg 1 (phantom row): a skills/** table row citing an identifier that greps to
zero in packages/** is a red. Leg 2 (missing row): a section REGISTERED as
exhaustive over a schema symbol that does not document every member is a red.
One corpus walk, two legs.
Leg 2 is table-driven rather than inferred, and that is a measurement result:
auto-binding a doc table to any enum it is a subset of scores 1 true positive
in 3 on this corpus. The heuristic ships as --suggest, which cannot fail.
Zero skills/** edits: the five surviving Leg 1 sites and the one measured Leg 2
gap are ledgered, not fixed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT
- check-ratchet-remedy-authority CONTROL row (the sweep classified it `marked`
before the row was written; the row is recorded from that verdict).
- bare-root-worklist: REFUSE-WIDE for IMPL_ROOTS `packages` — the word index
reads 5405 of 5640 tracked files (96%), so the population IS the root and a
true declaration would name this gate on every card touching any package.
- The watch-hint self-test asserts the spelling without writing a second
declaration site; spelling it as a regex literal made the real one
unjudgeable ("2 declaration sites").
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT
@github-actionsgithub-actionsBot added size/xl ci/cd dependencies Pull requests that update a dependency file labels Aug 31, 2026
@huangyiirenehuangyiirene added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 31, 2026 — with Claude
@huangyiirene
huangyiirene marked this pull request as ready for review August 31, 2026 16:41
@huangyiirene
huangyiirene added this pull request to the merge queueAug 31, 2026
Merged via the queue into main with commit ea0708cAug 31, 2026
37 checks passed
@huangyiirene
huangyiirene deleted the claude/issue-13678-skill-identifier-liveness-gate branch August 31, 2026 17:05
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cddependenciesPull requests that update a dependency filesize/xlskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants

@huangyiirene@claude
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Two-legged identifier-liveness gate over the published skills catalog (#13678) by huangyiirene · Pull Request #13892 · objectstack-ai/objectstack · GitHub
Skip to content

Two-legged identifier-liveness gate over the published skills catalog (#13678) - #13892

Merged
huangyiirene merged 2 commits into
mainfrom
claude/issue-13678-skill-identifier-liveness-gate
Aug 31, 2026
Merged

Two-legged identifier-liveness gate over the published skills catalog (#13678)#13892
huangyiirene merged 2 commits into
mainfrom
claude/issue-13678-skill-identifier-liveness-gate

Conversation

@huangyiirene

Copy link
Copy Markdown
Collaborator

Fixes#13678

A two-legged identifier-liveness gate over the published skills/** catalog, plus its self-test and its registrations with the farm's convention gates. Zero skills/** edits — the gate reads that corpus, it does not fix it.

The two predicates, stated precisely

Leg 1 — phantom row. A CITATION is a markdown table row, outside a fenced block, whose FIRST cell is exactly one backticked token of identifier shape (dotted segments, optional [] as in bands[].key), optionally followed by a short parenthetical annotation. It is LIVE when every dotted segment appears as a word token in the IMPLEMENTATION INDEX. Zero hits on any segment is a red naming file, line, identifier and the failing segment.

The index is built from packages/** source only — never from skills/** and never from content/docs/**. That exclusion is the load-bearing line in the leg: an index containing the teaching corpus would find every citation in the corpus that cites it, so every row would be live, the finding set would be empty on any tree, and the gate would be a green light wired to nothing. dist/ is skipped for the mirror-image reason — a stale build output keeps a retired identifier alive for as long as nobody cleans it, making the verdict a function of the developer's working directory. Both are --self-test assertions, not prose.

Three deliberate refusals in the extractor, each measured rather than assumed:

  • The trailing annotation.`none` (default) is a citation of none. A first cut required the cell to be exactly a backticked token; that omission silently dropped every annotated row, and on the Leg 2 side it manufactured two false reds by making complete tables read as 4-of-5 and 3-of-4.
  • The negative column. A migration table's left column cites the dead spelling BY DESIGN (| Legacy | CEL |, | Old | New |). Reddening those runs the gate backwards. Columns whose HEADER names the pre-migration or negative side are structurally out of the population — structurally, not by ledger entry, so those files never enter a ledger whose budget a later edit could spend.
  • The fence. Code fences are skipped. This is the opposite choice from check-corpus-claim-drift, deliberately: that gate exists because its defect lived in a comment inside a fence. This one reads table rows, and a pipe inside a fenced example is not one.

Leg 2 — missing row. A BINDING is a human assertion, in a table in the gate, that one named section of one file is exhaustive over one named schema symbol. For each binding: resolve the symbol from its declared source (z.enum or a named string-literal union, comments stripped first), locate the heading, take the section down to the next heading of the same-or-higher level, and require every member to appear inside some backticked span in that section. A missing member is a red.

Two structural safety rules: the anchor must be unique in its file (an ambiguous heading fails rather than silently first-matching a different table than the row's author read), and the symbol must resolve (a binding naming a deleted enum is STALE and fails — without that, deleting the schema would turn the row green, the exact inversion of what the row is for).

Scope is the SECTION, not the file: an exhaustiveness claim covers what a reader sees under that heading, and a file-wide pool credits a member mentioned 800 lines away. Measured, the difference is real — NavItemVariant reads 7-of-9 against its section and 8-of-9 against its whole file.

Why Leg 2 is registered rather than inferred — the false-red measurements

The scope note asks for a conservative "presented as exhaustive" predicate. Three candidates were built and measured on the real corpus before this shape was chosen.

PredicateRecallPrecisionVerdict
(a) auto-bind to any enum the table is a subset of17 bindings1 true positive in 3cannot fail CI
(b) keyword exhaustiveness markers ("all", "every", "the N types")18 of 155 tablespoor before member comparisonunreliable claim detector
(c) table names its own schema symbol in backticks1 of 155 tables, and that one hedgedhighno recall, and zero skills/** edits permitted

(a) in detail. 17 bindings, 9 candidate reds on the first cut. Six were artefacts: two from the trailing-annotation bug above (MetadataLockSchema read 3-of-4 and SeedMode 4-of-5 while both tables were complete), one from an enumeration split across several tables in one file (FlowNodeAction, whose own lead says "20 built-in node types"), and three from prose naming its own partiality. Of the three that survived every fix, two are one enumAggregationMetricType carries six aggregation functions plus three number/string/boolean members whose comments read "Custom SQL expression returning X". A table titled "Basic Aggregation Functions" listing exactly the six is CORRECT, and this predicate calls it false. In two files.

(b) in detail. The top-ranked hit is skills/objectstack-ai/SKILL.md "Three action types dispatch headlessly" — it matches on both "every" and a numeral and is a deliberate 3-of-6 subset. Most other matches belong to a neighbouring sentence, not the table ("every author must remember to re-attach", "all optional").

So the table-driven shape is a measurement result, not a preference, and it follows the idiom PR #13742 established next door: the engine takes the table as a parameter and --self-test drives synthetic rows through it, so genericity is proven without filling the table. The heuristic is not discarded — it ships as --suggest, a non-failing discovery mode that lists unregistered candidates, so the registry has a feeder and the numbers above stay reproducible instead of living only in a comment.

False-red analysis on the current corpus — every hit dispositioned

Leg 1: 500 citations over 48 published files, checked against 91,177 word tokens from 5,405 files. Five rows survive the predicate. None is a phantom, and they are two structural classes:

SiteIdentifierKindWhy it cannot be indexed here
objectstack-pm-dispatch/SKILL.md:98,102,103backlogRepo, conventionsFile, routingLabelPrefixskill-owned-configKeys of a config file the CUSTOMER writes, consumed by the skill's own prose. The skill IS the implementation; there is no repo symbol to grep.
objectstack-ui/SKILL.md:557,562dayStart, showMidnightsibling-repoLive in ../objectui at packages/plugin-gantt/src/GanttView.tsx. This repo ships backend only, packages/console/dist is gitignored, and sdui.manifest.json does not carry gantt plugin props — checked, not assumed.

A sixth, bare_field in objectstack-formula, sits under a | Legacy | CEL | header and is removed structurally by the negative-column rule rather than ledgered.

Leg 2: 9 registered bindings. Eight are complete today and are now pinned against future drift: ApiMethod, ObjectAccessScopeSchema, HookEvent (x2 files), LifecycleClassSchema, SeedMode, ReportType, ActionType. The ninth is a real, known gap, ledgered rather than fixed: ### Navigation Item Types documents 7 of the 9 NavItemVariant members — action and component have no row. Filed as #13890, a sub-issue of the sweep program.

Registering ActionType against the UI skill's complete table while the AI skill documents a deliberate 3-of-6 subset of the same enum is the registry earning its keep: a binding names a SECTION, so the correct table is pinned and the subset is left alone.

One finding worth a reviewer's eye, which the gate deliberately does not act on.ApproverType has ten members including role, and the doc documents nine. Documenting the tenth would trip check:role-word, which makes "role" a reserved-forbidden word across this same corpus. Two gates over one corpus can be put in direct conflict by a schema member's name. No binding is registered for it, and that is a decision rather than an oversight.

Robustness to the pending sweep merges

The brief asks that in-flight sweeps not redden this. Checked, not assumed: #13833 touches objectstack-i18n, #13861 touches objectstack-upgrade and adds skills/spec-property-retirement. Neither touches any file this gate ledgers. Predicates are identifier-level throughout — there is no count pin anywhere in either leg, so a sweep that adds or rewrites prose moves nothing unless it adds a citation that greps to zero, which is the gate working.

The two ledgers move in OPPOSITE directions, deliberately

  • Leg 1 exemptions — PRUNE-ONLY.kind and note are human judgements about why an identifier is unreachable from this repo; nothing in the tree encodes them, so an --update that rewrote this list would invent classifications it cannot know. --update only DELETES entries the scan no longer reaches. Shrink-only by construction, not by policy: there is no code path that grows it.
  • Leg 2 gaps — REWRITTEN FROM THE TREE, shrink-only in spirit. A gap is a measured count and is fully derivable, so --update moves it whichever way the tree moved. Only policy tells those apart, so the new-gap and grown-gap messages mark that path ⛔ MAINTAINER-ONLY while the author's own remedy is offered first and unmarked.

A gap that SHRANK fails and asks for --update — an improvement that does not move the ledger leaves budget behind for a later edit to spend silently.

Ablation — the gate CAN fail, on both legs

Commit-before-mutate: the implementation was committed first, so the restore leg points at HEAD and HEAD really holds it. Each leg proves its mutation reached DISK before any verdict is read (an editor's exit code proves nothing — sed/perl -i/replace all exit 0 on zero hits), and each restore is proven by blob-hash equality against the HEAD blob, never by an exit code. The script carries a trap ... EXIT INT TERM, restores with git checkout HEAD -- PATH (never the bare form, which restores from the index and would hand the mutation straight back at exit 0), and treats an empty hash as FAILURE rather than "nothing to compare".

=== BASELINE (unmutated tree) ===
exit=0
restored OK skills/objectstack-ui/SKILL.md blob=53121751928a… == HEAD:…
restored OK skills/objectstack-api/SKILL.md blob=c73e66997112… == HEAD:…
=== LEG 1 ABLATION: inject a PHANTOM ROW ===
injected-token count: 1 blob 53121751928a… -> c134f11ea385…
GATE exit=1
[leg1-phantom] skills/objectstack-ui/SKILL.md:642 cites `zzPhantomKeyNeverImplemented`
and the segment `zzPhantomKeyNeverImplemented` has ZERO hits in packages.
restored OK blob=53121751928a… == HEAD gate after restore exit=0
=== LEG 2 ABLATION: delete a documented MEMBER row ===
removed-row count now: 0 blob c73e66997112… -> 11673302ee99…
GATE exit=1
[leg2-missing-row] skills/objectstack-api/SKILL.md — ## API Methods (Operations) is
registered exhaustive over `ApiMethod` (packages/spec/src/data/object.zod.ts) and
does not document `bulk`.
restored OK blob=c73e66997112… == HEAD gate after restore exit=0

git status --porcelain is empty on the tree this PR was pushed from.

Registrations

  • check-ratchet-remedy-authority — a CONTROL row. The sweep classified this gate markedbefore the row was written, and the row is recorded from that verdict rather than from author intent; the corpus is set-equality audited both ways, so the row was demanded, not volunteered. Getting there needed the remedy text to carry real shrink testimony — the first wording said "weakens the gate", which the detector cannot anchor. Farm now: 12 marked, 6 refusing, 27 control rows.
  • bare-root-worklistREFUSE-WIDE for IMPL_ROOTS packages. The index reads 5,405 of 5,640 tracked files under that root (96%), so the population is not a subset of the root, it IS the root; a true declaration would name this gate on every card touching any package. Its other root, skills, IS declared beside the constant via ROOT_DIR_WATCH_HINTS — small, and the surface the gate exists for.
  • check-watch-hint-literal — the watch-hint self-test asserts the spelling without writing a second declaration site. Spelling the whole const NAME = [...] as a regex literal creates one, and the gate then reported "2 declaration sites — this gate cannot judge a declaration it cannot locate". Measured, then fixed.
  • lint.yml — one step in the job that carries the whole check:* family, with no paths filter. That matters more here than for its neighbours: the gate's two inputs are skills/** and packages/**, so a filter on either would let the OTHER side drift a member in or an identifier out without ever running the check that notices.

Verification

All readings taken on 1d84cf3b4, a clean tree, exit codes captured before any pipe.

node scripts/check-skill-identifier-liveness.mjs --self-test exit=0
node scripts/check-skill-identifier-liveness.mjs exit=0
Leg 1: 500 citations / 48 published files / 91,177 tokens / 5 exemptions
Leg 2: 9 registered exhaustive sections, 1 ledgered gap

Full derived family union from node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (33 families — all run, both invocation spellings), every one exit 0: entry-guard · parse-guard · watch-hint-literal · ratchet-remedy-authority · skill-identifier-liveness · agent-test-spelling · cli-command-ids · bash32-floor · pnpm-filter-targets · node-version · pnpm-acquisition · nul-bytes · cross-package-test-inputs · required-contexts · shard-attestation · stall-guard-budget · turbo-task-graph · workflow-status-functions · type-check-coverage · type-check-debt · dual-build-cjs-loads · pm-dispatch-gates · bare-root-worklist --self-test · aggregator-roster · ci-filter-parity · position-name-fold-loaders · self-test-wired · self-test-workflow-commands · step-collectors · whole-set-label-write · ci-failure --self-test · docs-audit/check-drift-comment.

Both convention-triggered obligations for adding a gate script were run and are green. check:type-check-debt was run after building the closure (turbo run build, 70/70 successful), because its --re-measure half silently measures a different world without it: OK — 29 ledger entries re-measured, 1531 raw tsc errors, none above its recorded number.

Repo-wide pnpm lint (eslint . --no-inline-config) run in full: exit 0 in 59s. No narrowing claimed.

node scripts/check-test-completeness.mjs returns exit 3 = PREREQUISITE NOT MET (it grades a saved turbo run test log that only CI produces). Recorded as NOT MEASURED — it is not a finding and not a failure.

The two edited gate scripts (check-ratchet-remedy-authority.mjs, bare-root-worklist.mjs) carry no vitest suite — repo-wide, no test file names either — so their --self-test IS their suite, and both were run green.

Changeset

None, and skip-changeset applied. Verified rather than assumed: the diff is scripts/**, .github/workflows/lint.yml, and one alias line in the private root manifest (@objectstack/spec-monorepo, private: true). Nothing under packages/, apps/ or examples/ changed, so this releases nothing — the workflow's own words for this case are "this PR edits a CI-internal script is the textbook skip-changeset case".

Scope

scripts/ and .github/workflows/lint.yml are not governed surfaces — regular code lane, opened as draft for the PM. Adjacent cards #13859 and #13835 are deliberately untouched. #13859 is adjacent but not subsumed: it is about a retirement-pin scanning one markdown file, a different instrument over a different question — this gate would not have caught it, because a withdrawn claim's sentence is semantics, and Leg 1 catches existence only.


Generated by Claude Code

…ills catalog
Leg 1 (phantom row): a skills/** table row citing an identifier that greps to
zero in packages/** is a red. Leg 2 (missing row): a section REGISTERED as
exhaustive over a schema symbol that does not document every member is a red.
One corpus walk, two legs.
Leg 2 is table-driven rather than inferred, and that is a measurement result:
auto-binding a doc table to any enum it is a subset of scores 1 true positive
in 3 on this corpus. The heuristic ships as --suggest, which cannot fail.
Zero skills/** edits: the five surviving Leg 1 sites and the one measured Leg 2
gap are ledgered, not fixed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT
- check-ratchet-remedy-authority CONTROL row (the sweep classified it `marked`
before the row was written; the row is recorded from that verdict).
- bare-root-worklist: REFUSE-WIDE for IMPL_ROOTS `packages` — the word index
reads 5405 of 5640 tracked files (96%), so the population IS the root and a
true declaration would name this gate on every card touching any package.
- The watch-hint self-test asserts the spelling without writing a second
declaration site; spelling it as a regex literal made the real one
unjudgeable ("2 declaration sites").
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT
@github-actionsgithub-actionsBot added size/xl ci/cd dependencies Pull requests that update a dependency file labels Aug 31, 2026
@huangyiirenehuangyiirene added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 31, 2026 — with Claude
@huangyiirene
huangyiirene marked this pull request as ready for review August 31, 2026 16:41
@huangyiirene
huangyiirene added this pull request to the merge queueAug 31, 2026
Merged via the queue into main with commit ea0708cAug 31, 2026
37 checks passed
@huangyiirene
huangyiirene deleted the claude/issue-13678-skill-identifier-liveness-gate branch August 31, 2026 17:05
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cddependenciesPull requests that update a dependency filesize/xlskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants

@huangyiirene@claude
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Two-legged identifier-liveness gate over the published skills catalog (#13678) by huangyiirene · Pull Request #13892 · objectstack-ai/objectstack · GitHub
Skip to content

Two-legged identifier-liveness gate over the published skills catalog (#13678) - #13892

Merged
huangyiirene merged 2 commits into
mainfrom
claude/issue-13678-skill-identifier-liveness-gate
Aug 31, 2026
Merged

Two-legged identifier-liveness gate over the published skills catalog (#13678)#13892
huangyiirene merged 2 commits into
mainfrom
claude/issue-13678-skill-identifier-liveness-gate

Conversation

@huangyiirene

Copy link
Copy Markdown
Collaborator

Fixes#13678

A two-legged identifier-liveness gate over the published skills/** catalog, plus its self-test and its registrations with the farm's convention gates. Zero skills/** edits — the gate reads that corpus, it does not fix it.

The two predicates, stated precisely

Leg 1 — phantom row. A CITATION is a markdown table row, outside a fenced block, whose FIRST cell is exactly one backticked token of identifier shape (dotted segments, optional [] as in bands[].key), optionally followed by a short parenthetical annotation. It is LIVE when every dotted segment appears as a word token in the IMPLEMENTATION INDEX. Zero hits on any segment is a red naming file, line, identifier and the failing segment.

The index is built from packages/** source only — never from skills/** and never from content/docs/**. That exclusion is the load-bearing line in the leg: an index containing the teaching corpus would find every citation in the corpus that cites it, so every row would be live, the finding set would be empty on any tree, and the gate would be a green light wired to nothing. dist/ is skipped for the mirror-image reason — a stale build output keeps a retired identifier alive for as long as nobody cleans it, making the verdict a function of the developer's working directory. Both are --self-test assertions, not prose.

Three deliberate refusals in the extractor, each measured rather than assumed:

  • The trailing annotation.`none` (default) is a citation of none. A first cut required the cell to be exactly a backticked token; that omission silently dropped every annotated row, and on the Leg 2 side it manufactured two false reds by making complete tables read as 4-of-5 and 3-of-4.
  • The negative column. A migration table's left column cites the dead spelling BY DESIGN (| Legacy | CEL |, | Old | New |). Reddening those runs the gate backwards. Columns whose HEADER names the pre-migration or negative side are structurally out of the population — structurally, not by ledger entry, so those files never enter a ledger whose budget a later edit could spend.
  • The fence. Code fences are skipped. This is the opposite choice from check-corpus-claim-drift, deliberately: that gate exists because its defect lived in a comment inside a fence. This one reads table rows, and a pipe inside a fenced example is not one.

Leg 2 — missing row. A BINDING is a human assertion, in a table in the gate, that one named section of one file is exhaustive over one named schema symbol. For each binding: resolve the symbol from its declared source (z.enum or a named string-literal union, comments stripped first), locate the heading, take the section down to the next heading of the same-or-higher level, and require every member to appear inside some backticked span in that section. A missing member is a red.

Two structural safety rules: the anchor must be unique in its file (an ambiguous heading fails rather than silently first-matching a different table than the row's author read), and the symbol must resolve (a binding naming a deleted enum is STALE and fails — without that, deleting the schema would turn the row green, the exact inversion of what the row is for).

Scope is the SECTION, not the file: an exhaustiveness claim covers what a reader sees under that heading, and a file-wide pool credits a member mentioned 800 lines away. Measured, the difference is real — NavItemVariant reads 7-of-9 against its section and 8-of-9 against its whole file.

Why Leg 2 is registered rather than inferred — the false-red measurements

The scope note asks for a conservative "presented as exhaustive" predicate. Three candidates were built and measured on the real corpus before this shape was chosen.

PredicateRecallPrecisionVerdict
(a) auto-bind to any enum the table is a subset of17 bindings1 true positive in 3cannot fail CI
(b) keyword exhaustiveness markers ("all", "every", "the N types")18 of 155 tablespoor before member comparisonunreliable claim detector
(c) table names its own schema symbol in backticks1 of 155 tables, and that one hedgedhighno recall, and zero skills/** edits permitted

(a) in detail. 17 bindings, 9 candidate reds on the first cut. Six were artefacts: two from the trailing-annotation bug above (MetadataLockSchema read 3-of-4 and SeedMode 4-of-5 while both tables were complete), one from an enumeration split across several tables in one file (FlowNodeAction, whose own lead says "20 built-in node types"), and three from prose naming its own partiality. Of the three that survived every fix, two are one enumAggregationMetricType carries six aggregation functions plus three number/string/boolean members whose comments read "Custom SQL expression returning X". A table titled "Basic Aggregation Functions" listing exactly the six is CORRECT, and this predicate calls it false. In two files.

(b) in detail. The top-ranked hit is skills/objectstack-ai/SKILL.md "Three action types dispatch headlessly" — it matches on both "every" and a numeral and is a deliberate 3-of-6 subset. Most other matches belong to a neighbouring sentence, not the table ("every author must remember to re-attach", "all optional").

So the table-driven shape is a measurement result, not a preference, and it follows the idiom PR #13742 established next door: the engine takes the table as a parameter and --self-test drives synthetic rows through it, so genericity is proven without filling the table. The heuristic is not discarded — it ships as --suggest, a non-failing discovery mode that lists unregistered candidates, so the registry has a feeder and the numbers above stay reproducible instead of living only in a comment.

False-red analysis on the current corpus — every hit dispositioned

Leg 1: 500 citations over 48 published files, checked against 91,177 word tokens from 5,405 files. Five rows survive the predicate. None is a phantom, and they are two structural classes:

SiteIdentifierKindWhy it cannot be indexed here
objectstack-pm-dispatch/SKILL.md:98,102,103backlogRepo, conventionsFile, routingLabelPrefixskill-owned-configKeys of a config file the CUSTOMER writes, consumed by the skill's own prose. The skill IS the implementation; there is no repo symbol to grep.
objectstack-ui/SKILL.md:557,562dayStart, showMidnightsibling-repoLive in ../objectui at packages/plugin-gantt/src/GanttView.tsx. This repo ships backend only, packages/console/dist is gitignored, and sdui.manifest.json does not carry gantt plugin props — checked, not assumed.

A sixth, bare_field in objectstack-formula, sits under a | Legacy | CEL | header and is removed structurally by the negative-column rule rather than ledgered.

Leg 2: 9 registered bindings. Eight are complete today and are now pinned against future drift: ApiMethod, ObjectAccessScopeSchema, HookEvent (x2 files), LifecycleClassSchema, SeedMode, ReportType, ActionType. The ninth is a real, known gap, ledgered rather than fixed: ### Navigation Item Types documents 7 of the 9 NavItemVariant members — action and component have no row. Filed as #13890, a sub-issue of the sweep program.

Registering ActionType against the UI skill's complete table while the AI skill documents a deliberate 3-of-6 subset of the same enum is the registry earning its keep: a binding names a SECTION, so the correct table is pinned and the subset is left alone.

One finding worth a reviewer's eye, which the gate deliberately does not act on.ApproverType has ten members including role, and the doc documents nine. Documenting the tenth would trip check:role-word, which makes "role" a reserved-forbidden word across this same corpus. Two gates over one corpus can be put in direct conflict by a schema member's name. No binding is registered for it, and that is a decision rather than an oversight.

Robustness to the pending sweep merges

The brief asks that in-flight sweeps not redden this. Checked, not assumed: #13833 touches objectstack-i18n, #13861 touches objectstack-upgrade and adds skills/spec-property-retirement. Neither touches any file this gate ledgers. Predicates are identifier-level throughout — there is no count pin anywhere in either leg, so a sweep that adds or rewrites prose moves nothing unless it adds a citation that greps to zero, which is the gate working.

The two ledgers move in OPPOSITE directions, deliberately

  • Leg 1 exemptions — PRUNE-ONLY.kind and note are human judgements about why an identifier is unreachable from this repo; nothing in the tree encodes them, so an --update that rewrote this list would invent classifications it cannot know. --update only DELETES entries the scan no longer reaches. Shrink-only by construction, not by policy: there is no code path that grows it.
  • Leg 2 gaps — REWRITTEN FROM THE TREE, shrink-only in spirit. A gap is a measured count and is fully derivable, so --update moves it whichever way the tree moved. Only policy tells those apart, so the new-gap and grown-gap messages mark that path ⛔ MAINTAINER-ONLY while the author's own remedy is offered first and unmarked.

A gap that SHRANK fails and asks for --update — an improvement that does not move the ledger leaves budget behind for a later edit to spend silently.

Ablation — the gate CAN fail, on both legs

Commit-before-mutate: the implementation was committed first, so the restore leg points at HEAD and HEAD really holds it. Each leg proves its mutation reached DISK before any verdict is read (an editor's exit code proves nothing — sed/perl -i/replace all exit 0 on zero hits), and each restore is proven by blob-hash equality against the HEAD blob, never by an exit code. The script carries a trap ... EXIT INT TERM, restores with git checkout HEAD -- PATH (never the bare form, which restores from the index and would hand the mutation straight back at exit 0), and treats an empty hash as FAILURE rather than "nothing to compare".

=== BASELINE (unmutated tree) ===
exit=0
restored OK skills/objectstack-ui/SKILL.md blob=53121751928a… == HEAD:…
restored OK skills/objectstack-api/SKILL.md blob=c73e66997112… == HEAD:…
=== LEG 1 ABLATION: inject a PHANTOM ROW ===
injected-token count: 1 blob 53121751928a… -> c134f11ea385…
GATE exit=1
[leg1-phantom] skills/objectstack-ui/SKILL.md:642 cites `zzPhantomKeyNeverImplemented`
and the segment `zzPhantomKeyNeverImplemented` has ZERO hits in packages.
restored OK blob=53121751928a… == HEAD gate after restore exit=0
=== LEG 2 ABLATION: delete a documented MEMBER row ===
removed-row count now: 0 blob c73e66997112… -> 11673302ee99…
GATE exit=1
[leg2-missing-row] skills/objectstack-api/SKILL.md — ## API Methods (Operations) is
registered exhaustive over `ApiMethod` (packages/spec/src/data/object.zod.ts) and
does not document `bulk`.
restored OK blob=c73e66997112… == HEAD gate after restore exit=0

git status --porcelain is empty on the tree this PR was pushed from.

Registrations

  • check-ratchet-remedy-authority — a CONTROL row. The sweep classified this gate markedbefore the row was written, and the row is recorded from that verdict rather than from author intent; the corpus is set-equality audited both ways, so the row was demanded, not volunteered. Getting there needed the remedy text to carry real shrink testimony — the first wording said "weakens the gate", which the detector cannot anchor. Farm now: 12 marked, 6 refusing, 27 control rows.
  • bare-root-worklistREFUSE-WIDE for IMPL_ROOTS packages. The index reads 5,405 of 5,640 tracked files under that root (96%), so the population is not a subset of the root, it IS the root; a true declaration would name this gate on every card touching any package. Its other root, skills, IS declared beside the constant via ROOT_DIR_WATCH_HINTS — small, and the surface the gate exists for.
  • check-watch-hint-literal — the watch-hint self-test asserts the spelling without writing a second declaration site. Spelling the whole const NAME = [...] as a regex literal creates one, and the gate then reported "2 declaration sites — this gate cannot judge a declaration it cannot locate". Measured, then fixed.
  • lint.yml — one step in the job that carries the whole check:* family, with no paths filter. That matters more here than for its neighbours: the gate's two inputs are skills/** and packages/**, so a filter on either would let the OTHER side drift a member in or an identifier out without ever running the check that notices.

Verification

All readings taken on 1d84cf3b4, a clean tree, exit codes captured before any pipe.

node scripts/check-skill-identifier-liveness.mjs --self-test exit=0
node scripts/check-skill-identifier-liveness.mjs exit=0
Leg 1: 500 citations / 48 published files / 91,177 tokens / 5 exemptions
Leg 2: 9 registered exhaustive sections, 1 ledgered gap

Full derived family union from node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (33 families — all run, both invocation spellings), every one exit 0: entry-guard · parse-guard · watch-hint-literal · ratchet-remedy-authority · skill-identifier-liveness · agent-test-spelling · cli-command-ids · bash32-floor · pnpm-filter-targets · node-version · pnpm-acquisition · nul-bytes · cross-package-test-inputs · required-contexts · shard-attestation · stall-guard-budget · turbo-task-graph · workflow-status-functions · type-check-coverage · type-check-debt · dual-build-cjs-loads · pm-dispatch-gates · bare-root-worklist --self-test · aggregator-roster · ci-filter-parity · position-name-fold-loaders · self-test-wired · self-test-workflow-commands · step-collectors · whole-set-label-write · ci-failure --self-test · docs-audit/check-drift-comment.

Both convention-triggered obligations for adding a gate script were run and are green. check:type-check-debt was run after building the closure (turbo run build, 70/70 successful), because its --re-measure half silently measures a different world without it: OK — 29 ledger entries re-measured, 1531 raw tsc errors, none above its recorded number.

Repo-wide pnpm lint (eslint . --no-inline-config) run in full: exit 0 in 59s. No narrowing claimed.

node scripts/check-test-completeness.mjs returns exit 3 = PREREQUISITE NOT MET (it grades a saved turbo run test log that only CI produces). Recorded as NOT MEASURED — it is not a finding and not a failure.

The two edited gate scripts (check-ratchet-remedy-authority.mjs, bare-root-worklist.mjs) carry no vitest suite — repo-wide, no test file names either — so their --self-test IS their suite, and both were run green.

Changeset

None, and skip-changeset applied. Verified rather than assumed: the diff is scripts/**, .github/workflows/lint.yml, and one alias line in the private root manifest (@objectstack/spec-monorepo, private: true). Nothing under packages/, apps/ or examples/ changed, so this releases nothing — the workflow's own words for this case are "this PR edits a CI-internal script is the textbook skip-changeset case".

Scope

scripts/ and .github/workflows/lint.yml are not governed surfaces — regular code lane, opened as draft for the PM. Adjacent cards #13859 and #13835 are deliberately untouched. #13859 is adjacent but not subsumed: it is about a retirement-pin scanning one markdown file, a different instrument over a different question — this gate would not have caught it, because a withdrawn claim's sentence is semantics, and Leg 1 catches existence only.


Generated by Claude Code

…ills catalog
Leg 1 (phantom row): a skills/** table row citing an identifier that greps to
zero in packages/** is a red. Leg 2 (missing row): a section REGISTERED as
exhaustive over a schema symbol that does not document every member is a red.
One corpus walk, two legs.
Leg 2 is table-driven rather than inferred, and that is a measurement result:
auto-binding a doc table to any enum it is a subset of scores 1 true positive
in 3 on this corpus. The heuristic ships as --suggest, which cannot fail.
Zero skills/** edits: the five surviving Leg 1 sites and the one measured Leg 2
gap are ledgered, not fixed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT
- check-ratchet-remedy-authority CONTROL row (the sweep classified it `marked`
before the row was written; the row is recorded from that verdict).
- bare-root-worklist: REFUSE-WIDE for IMPL_ROOTS `packages` — the word index
reads 5405 of 5640 tracked files (96%), so the population IS the root and a
true declaration would name this gate on every card touching any package.
- The watch-hint self-test asserts the spelling without writing a second
declaration site; spelling it as a regex literal made the real one
unjudgeable ("2 declaration sites").
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT
@github-actionsgithub-actionsBot added size/xl ci/cd dependencies Pull requests that update a dependency file labels Aug 31, 2026
@huangyiirenehuangyiirene added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 31, 2026 — with Claude
@huangyiirene
huangyiirene marked this pull request as ready for review August 31, 2026 16:41
@huangyiirene
huangyiirene added this pull request to the merge queueAug 31, 2026
Merged via the queue into main with commit ea0708cAug 31, 2026
37 checks passed
@huangyiirene
huangyiirene deleted the claude/issue-13678-skill-identifier-liveness-gate branch August 31, 2026 17:05
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cddependenciesPull requests that update a dependency filesize/xlskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants

@huangyiirene@claude
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Two-legged identifier-liveness gate over the published skills catalog (#13678) by huangyiirene · Pull Request #13892 · objectstack-ai/objectstack · GitHub
Skip to content

Two-legged identifier-liveness gate over the published skills catalog (#13678) - #13892

Merged
huangyiirene merged 2 commits into
mainfrom
claude/issue-13678-skill-identifier-liveness-gate
Aug 31, 2026
Merged

Two-legged identifier-liveness gate over the published skills catalog (#13678)#13892
huangyiirene merged 2 commits into
mainfrom
claude/issue-13678-skill-identifier-liveness-gate

Conversation

@huangyiirene

Copy link
Copy Markdown
Collaborator

Fixes#13678

A two-legged identifier-liveness gate over the published skills/** catalog, plus its self-test and its registrations with the farm's convention gates. Zero skills/** edits — the gate reads that corpus, it does not fix it.

The two predicates, stated precisely

Leg 1 — phantom row. A CITATION is a markdown table row, outside a fenced block, whose FIRST cell is exactly one backticked token of identifier shape (dotted segments, optional [] as in bands[].key), optionally followed by a short parenthetical annotation. It is LIVE when every dotted segment appears as a word token in the IMPLEMENTATION INDEX. Zero hits on any segment is a red naming file, line, identifier and the failing segment.

The index is built from packages/** source only — never from skills/** and never from content/docs/**. That exclusion is the load-bearing line in the leg: an index containing the teaching corpus would find every citation in the corpus that cites it, so every row would be live, the finding set would be empty on any tree, and the gate would be a green light wired to nothing. dist/ is skipped for the mirror-image reason — a stale build output keeps a retired identifier alive for as long as nobody cleans it, making the verdict a function of the developer's working directory. Both are --self-test assertions, not prose.

Three deliberate refusals in the extractor, each measured rather than assumed:

  • The trailing annotation.`none` (default) is a citation of none. A first cut required the cell to be exactly a backticked token; that omission silently dropped every annotated row, and on the Leg 2 side it manufactured two false reds by making complete tables read as 4-of-5 and 3-of-4.
  • The negative column. A migration table's left column cites the dead spelling BY DESIGN (| Legacy | CEL |, | Old | New |). Reddening those runs the gate backwards. Columns whose HEADER names the pre-migration or negative side are structurally out of the population — structurally, not by ledger entry, so those files never enter a ledger whose budget a later edit could spend.
  • The fence. Code fences are skipped. This is the opposite choice from check-corpus-claim-drift, deliberately: that gate exists because its defect lived in a comment inside a fence. This one reads table rows, and a pipe inside a fenced example is not one.

Leg 2 — missing row. A BINDING is a human assertion, in a table in the gate, that one named section of one file is exhaustive over one named schema symbol. For each binding: resolve the symbol from its declared source (z.enum or a named string-literal union, comments stripped first), locate the heading, take the section down to the next heading of the same-or-higher level, and require every member to appear inside some backticked span in that section. A missing member is a red.

Two structural safety rules: the anchor must be unique in its file (an ambiguous heading fails rather than silently first-matching a different table than the row's author read), and the symbol must resolve (a binding naming a deleted enum is STALE and fails — without that, deleting the schema would turn the row green, the exact inversion of what the row is for).

Scope is the SECTION, not the file: an exhaustiveness claim covers what a reader sees under that heading, and a file-wide pool credits a member mentioned 800 lines away. Measured, the difference is real — NavItemVariant reads 7-of-9 against its section and 8-of-9 against its whole file.

Why Leg 2 is registered rather than inferred — the false-red measurements

The scope note asks for a conservative "presented as exhaustive" predicate. Three candidates were built and measured on the real corpus before this shape was chosen.

PredicateRecallPrecisionVerdict
(a) auto-bind to any enum the table is a subset of17 bindings1 true positive in 3cannot fail CI
(b) keyword exhaustiveness markers ("all", "every", "the N types")18 of 155 tablespoor before member comparisonunreliable claim detector
(c) table names its own schema symbol in backticks1 of 155 tables, and that one hedgedhighno recall, and zero skills/** edits permitted

(a) in detail. 17 bindings, 9 candidate reds on the first cut. Six were artefacts: two from the trailing-annotation bug above (MetadataLockSchema read 3-of-4 and SeedMode 4-of-5 while both tables were complete), one from an enumeration split across several tables in one file (FlowNodeAction, whose own lead says "20 built-in node types"), and three from prose naming its own partiality. Of the three that survived every fix, two are one enumAggregationMetricType carries six aggregation functions plus three number/string/boolean members whose comments read "Custom SQL expression returning X". A table titled "Basic Aggregation Functions" listing exactly the six is CORRECT, and this predicate calls it false. In two files.

(b) in detail. The top-ranked hit is skills/objectstack-ai/SKILL.md "Three action types dispatch headlessly" — it matches on both "every" and a numeral and is a deliberate 3-of-6 subset. Most other matches belong to a neighbouring sentence, not the table ("every author must remember to re-attach", "all optional").

So the table-driven shape is a measurement result, not a preference, and it follows the idiom PR #13742 established next door: the engine takes the table as a parameter and --self-test drives synthetic rows through it, so genericity is proven without filling the table. The heuristic is not discarded — it ships as --suggest, a non-failing discovery mode that lists unregistered candidates, so the registry has a feeder and the numbers above stay reproducible instead of living only in a comment.

False-red analysis on the current corpus — every hit dispositioned

Leg 1: 500 citations over 48 published files, checked against 91,177 word tokens from 5,405 files. Five rows survive the predicate. None is a phantom, and they are two structural classes:

SiteIdentifierKindWhy it cannot be indexed here
objectstack-pm-dispatch/SKILL.md:98,102,103backlogRepo, conventionsFile, routingLabelPrefixskill-owned-configKeys of a config file the CUSTOMER writes, consumed by the skill's own prose. The skill IS the implementation; there is no repo symbol to grep.
objectstack-ui/SKILL.md:557,562dayStart, showMidnightsibling-repoLive in ../objectui at packages/plugin-gantt/src/GanttView.tsx. This repo ships backend only, packages/console/dist is gitignored, and sdui.manifest.json does not carry gantt plugin props — checked, not assumed.

A sixth, bare_field in objectstack-formula, sits under a | Legacy | CEL | header and is removed structurally by the negative-column rule rather than ledgered.

Leg 2: 9 registered bindings. Eight are complete today and are now pinned against future drift: ApiMethod, ObjectAccessScopeSchema, HookEvent (x2 files), LifecycleClassSchema, SeedMode, ReportType, ActionType. The ninth is a real, known gap, ledgered rather than fixed: ### Navigation Item Types documents 7 of the 9 NavItemVariant members — action and component have no row. Filed as #13890, a sub-issue of the sweep program.

Registering ActionType against the UI skill's complete table while the AI skill documents a deliberate 3-of-6 subset of the same enum is the registry earning its keep: a binding names a SECTION, so the correct table is pinned and the subset is left alone.

One finding worth a reviewer's eye, which the gate deliberately does not act on.ApproverType has ten members including role, and the doc documents nine. Documenting the tenth would trip check:role-word, which makes "role" a reserved-forbidden word across this same corpus. Two gates over one corpus can be put in direct conflict by a schema member's name. No binding is registered for it, and that is a decision rather than an oversight.

Robustness to the pending sweep merges

The brief asks that in-flight sweeps not redden this. Checked, not assumed: #13833 touches objectstack-i18n, #13861 touches objectstack-upgrade and adds skills/spec-property-retirement. Neither touches any file this gate ledgers. Predicates are identifier-level throughout — there is no count pin anywhere in either leg, so a sweep that adds or rewrites prose moves nothing unless it adds a citation that greps to zero, which is the gate working.

The two ledgers move in OPPOSITE directions, deliberately

  • Leg 1 exemptions — PRUNE-ONLY.kind and note are human judgements about why an identifier is unreachable from this repo; nothing in the tree encodes them, so an --update that rewrote this list would invent classifications it cannot know. --update only DELETES entries the scan no longer reaches. Shrink-only by construction, not by policy: there is no code path that grows it.
  • Leg 2 gaps — REWRITTEN FROM THE TREE, shrink-only in spirit. A gap is a measured count and is fully derivable, so --update moves it whichever way the tree moved. Only policy tells those apart, so the new-gap and grown-gap messages mark that path ⛔ MAINTAINER-ONLY while the author's own remedy is offered first and unmarked.

A gap that SHRANK fails and asks for --update — an improvement that does not move the ledger leaves budget behind for a later edit to spend silently.

Ablation — the gate CAN fail, on both legs

Commit-before-mutate: the implementation was committed first, so the restore leg points at HEAD and HEAD really holds it. Each leg proves its mutation reached DISK before any verdict is read (an editor's exit code proves nothing — sed/perl -i/replace all exit 0 on zero hits), and each restore is proven by blob-hash equality against the HEAD blob, never by an exit code. The script carries a trap ... EXIT INT TERM, restores with git checkout HEAD -- PATH (never the bare form, which restores from the index and would hand the mutation straight back at exit 0), and treats an empty hash as FAILURE rather than "nothing to compare".

=== BASELINE (unmutated tree) ===
exit=0
restored OK skills/objectstack-ui/SKILL.md blob=53121751928a… == HEAD:…
restored OK skills/objectstack-api/SKILL.md blob=c73e66997112… == HEAD:…
=== LEG 1 ABLATION: inject a PHANTOM ROW ===
injected-token count: 1 blob 53121751928a… -> c134f11ea385…
GATE exit=1
[leg1-phantom] skills/objectstack-ui/SKILL.md:642 cites `zzPhantomKeyNeverImplemented`
and the segment `zzPhantomKeyNeverImplemented` has ZERO hits in packages.
restored OK blob=53121751928a… == HEAD gate after restore exit=0
=== LEG 2 ABLATION: delete a documented MEMBER row ===
removed-row count now: 0 blob c73e66997112… -> 11673302ee99…
GATE exit=1
[leg2-missing-row] skills/objectstack-api/SKILL.md — ## API Methods (Operations) is
registered exhaustive over `ApiMethod` (packages/spec/src/data/object.zod.ts) and
does not document `bulk`.
restored OK blob=c73e66997112… == HEAD gate after restore exit=0

git status --porcelain is empty on the tree this PR was pushed from.

Registrations

  • check-ratchet-remedy-authority — a CONTROL row. The sweep classified this gate markedbefore the row was written, and the row is recorded from that verdict rather than from author intent; the corpus is set-equality audited both ways, so the row was demanded, not volunteered. Getting there needed the remedy text to carry real shrink testimony — the first wording said "weakens the gate", which the detector cannot anchor. Farm now: 12 marked, 6 refusing, 27 control rows.
  • bare-root-worklistREFUSE-WIDE for IMPL_ROOTS packages. The index reads 5,405 of 5,640 tracked files under that root (96%), so the population is not a subset of the root, it IS the root; a true declaration would name this gate on every card touching any package. Its other root, skills, IS declared beside the constant via ROOT_DIR_WATCH_HINTS — small, and the surface the gate exists for.
  • check-watch-hint-literal — the watch-hint self-test asserts the spelling without writing a second declaration site. Spelling the whole const NAME = [...] as a regex literal creates one, and the gate then reported "2 declaration sites — this gate cannot judge a declaration it cannot locate". Measured, then fixed.
  • lint.yml — one step in the job that carries the whole check:* family, with no paths filter. That matters more here than for its neighbours: the gate's two inputs are skills/** and packages/**, so a filter on either would let the OTHER side drift a member in or an identifier out without ever running the check that notices.

Verification

All readings taken on 1d84cf3b4, a clean tree, exit codes captured before any pipe.

node scripts/check-skill-identifier-liveness.mjs --self-test exit=0
node scripts/check-skill-identifier-liveness.mjs exit=0
Leg 1: 500 citations / 48 published files / 91,177 tokens / 5 exemptions
Leg 2: 9 registered exhaustive sections, 1 ledgered gap

Full derived family union from node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (33 families — all run, both invocation spellings), every one exit 0: entry-guard · parse-guard · watch-hint-literal · ratchet-remedy-authority · skill-identifier-liveness · agent-test-spelling · cli-command-ids · bash32-floor · pnpm-filter-targets · node-version · pnpm-acquisition · nul-bytes · cross-package-test-inputs · required-contexts · shard-attestation · stall-guard-budget · turbo-task-graph · workflow-status-functions · type-check-coverage · type-check-debt · dual-build-cjs-loads · pm-dispatch-gates · bare-root-worklist --self-test · aggregator-roster · ci-filter-parity · position-name-fold-loaders · self-test-wired · self-test-workflow-commands · step-collectors · whole-set-label-write · ci-failure --self-test · docs-audit/check-drift-comment.

Both convention-triggered obligations for adding a gate script were run and are green. check:type-check-debt was run after building the closure (turbo run build, 70/70 successful), because its --re-measure half silently measures a different world without it: OK — 29 ledger entries re-measured, 1531 raw tsc errors, none above its recorded number.

Repo-wide pnpm lint (eslint . --no-inline-config) run in full: exit 0 in 59s. No narrowing claimed.

node scripts/check-test-completeness.mjs returns exit 3 = PREREQUISITE NOT MET (it grades a saved turbo run test log that only CI produces). Recorded as NOT MEASURED — it is not a finding and not a failure.

The two edited gate scripts (check-ratchet-remedy-authority.mjs, bare-root-worklist.mjs) carry no vitest suite — repo-wide, no test file names either — so their --self-test IS their suite, and both were run green.

Changeset

None, and skip-changeset applied. Verified rather than assumed: the diff is scripts/**, .github/workflows/lint.yml, and one alias line in the private root manifest (@objectstack/spec-monorepo, private: true). Nothing under packages/, apps/ or examples/ changed, so this releases nothing — the workflow's own words for this case are "this PR edits a CI-internal script is the textbook skip-changeset case".

Scope

scripts/ and .github/workflows/lint.yml are not governed surfaces — regular code lane, opened as draft for the PM. Adjacent cards #13859 and #13835 are deliberately untouched. #13859 is adjacent but not subsumed: it is about a retirement-pin scanning one markdown file, a different instrument over a different question — this gate would not have caught it, because a withdrawn claim's sentence is semantics, and Leg 1 catches existence only.


Generated by Claude Code

…ills catalog
Leg 1 (phantom row): a skills/** table row citing an identifier that greps to
zero in packages/** is a red. Leg 2 (missing row): a section REGISTERED as
exhaustive over a schema symbol that does not document every member is a red.
One corpus walk, two legs.
Leg 2 is table-driven rather than inferred, and that is a measurement result:
auto-binding a doc table to any enum it is a subset of scores 1 true positive
in 3 on this corpus. The heuristic ships as --suggest, which cannot fail.
Zero skills/** edits: the five surviving Leg 1 sites and the one measured Leg 2
gap are ledgered, not fixed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT
- check-ratchet-remedy-authority CONTROL row (the sweep classified it `marked`
before the row was written; the row is recorded from that verdict).
- bare-root-worklist: REFUSE-WIDE for IMPL_ROOTS `packages` — the word index
reads 5405 of 5640 tracked files (96%), so the population IS the root and a
true declaration would name this gate on every card touching any package.
- The watch-hint self-test asserts the spelling without writing a second
declaration site; spelling it as a regex literal made the real one
unjudgeable ("2 declaration sites").
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT
@github-actionsgithub-actionsBot added size/xl ci/cd dependencies Pull requests that update a dependency file labels Aug 31, 2026
@huangyiirenehuangyiirene added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 31, 2026 — with Claude
@huangyiirene
huangyiirene marked this pull request as ready for review August 31, 2026 16:41
@huangyiirene
huangyiirene added this pull request to the merge queueAug 31, 2026
Merged via the queue into main with commit ea0708cAug 31, 2026
37 checks passed
@huangyiirene
huangyiirene deleted the claude/issue-13678-skill-identifier-liveness-gate branch August 31, 2026 17:05
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cddependenciesPull requests that update a dependency filesize/xlskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants

@huangyiirene@claude
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' Two-legged identifier-liveness gate over the published skills catalog (#13678) by huangyiirene · Pull Request #13892 · objectstack-ai/objectstack · GitHub
Skip to content

Two-legged identifier-liveness gate over the published skills catalog (#13678) - #13892

Merged
huangyiirene merged 2 commits into
mainfrom
claude/issue-13678-skill-identifier-liveness-gate
Aug 31, 2026
Merged

Two-legged identifier-liveness gate over the published skills catalog (#13678)#13892
huangyiirene merged 2 commits into
mainfrom
claude/issue-13678-skill-identifier-liveness-gate

Conversation

@huangyiirene

Copy link
Copy Markdown
Collaborator

Fixes#13678

A two-legged identifier-liveness gate over the published skills/** catalog, plus its self-test and its registrations with the farm's convention gates. Zero skills/** edits — the gate reads that corpus, it does not fix it.

The two predicates, stated precisely

Leg 1 — phantom row. A CITATION is a markdown table row, outside a fenced block, whose FIRST cell is exactly one backticked token of identifier shape (dotted segments, optional [] as in bands[].key), optionally followed by a short parenthetical annotation. It is LIVE when every dotted segment appears as a word token in the IMPLEMENTATION INDEX. Zero hits on any segment is a red naming file, line, identifier and the failing segment.

The index is built from packages/** source only — never from skills/** and never from content/docs/**. That exclusion is the load-bearing line in the leg: an index containing the teaching corpus would find every citation in the corpus that cites it, so every row would be live, the finding set would be empty on any tree, and the gate would be a green light wired to nothing. dist/ is skipped for the mirror-image reason — a stale build output keeps a retired identifier alive for as long as nobody cleans it, making the verdict a function of the developer's working directory. Both are --self-test assertions, not prose.

Three deliberate refusals in the extractor, each measured rather than assumed:

  • The trailing annotation.`none` (default) is a citation of none. A first cut required the cell to be exactly a backticked token; that omission silently dropped every annotated row, and on the Leg 2 side it manufactured two false reds by making complete tables read as 4-of-5 and 3-of-4.
  • The negative column. A migration table's left column cites the dead spelling BY DESIGN (| Legacy | CEL |, | Old | New |). Reddening those runs the gate backwards. Columns whose HEADER names the pre-migration or negative side are structurally out of the population — structurally, not by ledger entry, so those files never enter a ledger whose budget a later edit could spend.
  • The fence. Code fences are skipped. This is the opposite choice from check-corpus-claim-drift, deliberately: that gate exists because its defect lived in a comment inside a fence. This one reads table rows, and a pipe inside a fenced example is not one.

Leg 2 — missing row. A BINDING is a human assertion, in a table in the gate, that one named section of one file is exhaustive over one named schema symbol. For each binding: resolve the symbol from its declared source (z.enum or a named string-literal union, comments stripped first), locate the heading, take the section down to the next heading of the same-or-higher level, and require every member to appear inside some backticked span in that section. A missing member is a red.

Two structural safety rules: the anchor must be unique in its file (an ambiguous heading fails rather than silently first-matching a different table than the row's author read), and the symbol must resolve (a binding naming a deleted enum is STALE and fails — without that, deleting the schema would turn the row green, the exact inversion of what the row is for).

Scope is the SECTION, not the file: an exhaustiveness claim covers what a reader sees under that heading, and a file-wide pool credits a member mentioned 800 lines away. Measured, the difference is real — NavItemVariant reads 7-of-9 against its section and 8-of-9 against its whole file.

Why Leg 2 is registered rather than inferred — the false-red measurements

The scope note asks for a conservative "presented as exhaustive" predicate. Three candidates were built and measured on the real corpus before this shape was chosen.

PredicateRecallPrecisionVerdict
(a) auto-bind to any enum the table is a subset of17 bindings1 true positive in 3cannot fail CI
(b) keyword exhaustiveness markers ("all", "every", "the N types")18 of 155 tablespoor before member comparisonunreliable claim detector
(c) table names its own schema symbol in backticks1 of 155 tables, and that one hedgedhighno recall, and zero skills/** edits permitted

(a) in detail. 17 bindings, 9 candidate reds on the first cut. Six were artefacts: two from the trailing-annotation bug above (MetadataLockSchema read 3-of-4 and SeedMode 4-of-5 while both tables were complete), one from an enumeration split across several tables in one file (FlowNodeAction, whose own lead says "20 built-in node types"), and three from prose naming its own partiality. Of the three that survived every fix, two are one enumAggregationMetricType carries six aggregation functions plus three number/string/boolean members whose comments read "Custom SQL expression returning X". A table titled "Basic Aggregation Functions" listing exactly the six is CORRECT, and this predicate calls it false. In two files.

(b) in detail. The top-ranked hit is skills/objectstack-ai/SKILL.md "Three action types dispatch headlessly" — it matches on both "every" and a numeral and is a deliberate 3-of-6 subset. Most other matches belong to a neighbouring sentence, not the table ("every author must remember to re-attach", "all optional").

So the table-driven shape is a measurement result, not a preference, and it follows the idiom PR #13742 established next door: the engine takes the table as a parameter and --self-test drives synthetic rows through it, so genericity is proven without filling the table. The heuristic is not discarded — it ships as --suggest, a non-failing discovery mode that lists unregistered candidates, so the registry has a feeder and the numbers above stay reproducible instead of living only in a comment.

False-red analysis on the current corpus — every hit dispositioned

Leg 1: 500 citations over 48 published files, checked against 91,177 word tokens from 5,405 files. Five rows survive the predicate. None is a phantom, and they are two structural classes:

SiteIdentifierKindWhy it cannot be indexed here
objectstack-pm-dispatch/SKILL.md:98,102,103backlogRepo, conventionsFile, routingLabelPrefixskill-owned-configKeys of a config file the CUSTOMER writes, consumed by the skill's own prose. The skill IS the implementation; there is no repo symbol to grep.
objectstack-ui/SKILL.md:557,562dayStart, showMidnightsibling-repoLive in ../objectui at packages/plugin-gantt/src/GanttView.tsx. This repo ships backend only, packages/console/dist is gitignored, and sdui.manifest.json does not carry gantt plugin props — checked, not assumed.

A sixth, bare_field in objectstack-formula, sits under a | Legacy | CEL | header and is removed structurally by the negative-column rule rather than ledgered.

Leg 2: 9 registered bindings. Eight are complete today and are now pinned against future drift: ApiMethod, ObjectAccessScopeSchema, HookEvent (x2 files), LifecycleClassSchema, SeedMode, ReportType, ActionType. The ninth is a real, known gap, ledgered rather than fixed: ### Navigation Item Types documents 7 of the 9 NavItemVariant members — action and component have no row. Filed as #13890, a sub-issue of the sweep program.

Registering ActionType against the UI skill's complete table while the AI skill documents a deliberate 3-of-6 subset of the same enum is the registry earning its keep: a binding names a SECTION, so the correct table is pinned and the subset is left alone.

One finding worth a reviewer's eye, which the gate deliberately does not act on.ApproverType has ten members including role, and the doc documents nine. Documenting the tenth would trip check:role-word, which makes "role" a reserved-forbidden word across this same corpus. Two gates over one corpus can be put in direct conflict by a schema member's name. No binding is registered for it, and that is a decision rather than an oversight.

Robustness to the pending sweep merges

The brief asks that in-flight sweeps not redden this. Checked, not assumed: #13833 touches objectstack-i18n, #13861 touches objectstack-upgrade and adds skills/spec-property-retirement. Neither touches any file this gate ledgers. Predicates are identifier-level throughout — there is no count pin anywhere in either leg, so a sweep that adds or rewrites prose moves nothing unless it adds a citation that greps to zero, which is the gate working.

The two ledgers move in OPPOSITE directions, deliberately

  • Leg 1 exemptions — PRUNE-ONLY.kind and note are human judgements about why an identifier is unreachable from this repo; nothing in the tree encodes them, so an --update that rewrote this list would invent classifications it cannot know. --update only DELETES entries the scan no longer reaches. Shrink-only by construction, not by policy: there is no code path that grows it.
  • Leg 2 gaps — REWRITTEN FROM THE TREE, shrink-only in spirit. A gap is a measured count and is fully derivable, so --update moves it whichever way the tree moved. Only policy tells those apart, so the new-gap and grown-gap messages mark that path ⛔ MAINTAINER-ONLY while the author's own remedy is offered first and unmarked.

A gap that SHRANK fails and asks for --update — an improvement that does not move the ledger leaves budget behind for a later edit to spend silently.

Ablation — the gate CAN fail, on both legs

Commit-before-mutate: the implementation was committed first, so the restore leg points at HEAD and HEAD really holds it. Each leg proves its mutation reached DISK before any verdict is read (an editor's exit code proves nothing — sed/perl -i/replace all exit 0 on zero hits), and each restore is proven by blob-hash equality against the HEAD blob, never by an exit code. The script carries a trap ... EXIT INT TERM, restores with git checkout HEAD -- PATH (never the bare form, which restores from the index and would hand the mutation straight back at exit 0), and treats an empty hash as FAILURE rather than "nothing to compare".

=== BASELINE (unmutated tree) ===
exit=0
restored OK skills/objectstack-ui/SKILL.md blob=53121751928a… == HEAD:…
restored OK skills/objectstack-api/SKILL.md blob=c73e66997112… == HEAD:…
=== LEG 1 ABLATION: inject a PHANTOM ROW ===
injected-token count: 1 blob 53121751928a… -> c134f11ea385…
GATE exit=1
[leg1-phantom] skills/objectstack-ui/SKILL.md:642 cites `zzPhantomKeyNeverImplemented`
and the segment `zzPhantomKeyNeverImplemented` has ZERO hits in packages.
restored OK blob=53121751928a… == HEAD gate after restore exit=0
=== LEG 2 ABLATION: delete a documented MEMBER row ===
removed-row count now: 0 blob c73e66997112… -> 11673302ee99…
GATE exit=1
[leg2-missing-row] skills/objectstack-api/SKILL.md — ## API Methods (Operations) is
registered exhaustive over `ApiMethod` (packages/spec/src/data/object.zod.ts) and
does not document `bulk`.
restored OK blob=c73e66997112… == HEAD gate after restore exit=0

git status --porcelain is empty on the tree this PR was pushed from.

Registrations

  • check-ratchet-remedy-authority — a CONTROL row. The sweep classified this gate markedbefore the row was written, and the row is recorded from that verdict rather than from author intent; the corpus is set-equality audited both ways, so the row was demanded, not volunteered. Getting there needed the remedy text to carry real shrink testimony — the first wording said "weakens the gate", which the detector cannot anchor. Farm now: 12 marked, 6 refusing, 27 control rows.
  • bare-root-worklistREFUSE-WIDE for IMPL_ROOTS packages. The index reads 5,405 of 5,640 tracked files under that root (96%), so the population is not a subset of the root, it IS the root; a true declaration would name this gate on every card touching any package. Its other root, skills, IS declared beside the constant via ROOT_DIR_WATCH_HINTS — small, and the surface the gate exists for.
  • check-watch-hint-literal — the watch-hint self-test asserts the spelling without writing a second declaration site. Spelling the whole const NAME = [...] as a regex literal creates one, and the gate then reported "2 declaration sites — this gate cannot judge a declaration it cannot locate". Measured, then fixed.
  • lint.yml — one step in the job that carries the whole check:* family, with no paths filter. That matters more here than for its neighbours: the gate's two inputs are skills/** and packages/**, so a filter on either would let the OTHER side drift a member in or an identifier out without ever running the check that notices.

Verification

All readings taken on 1d84cf3b4, a clean tree, exit codes captured before any pipe.

node scripts/check-skill-identifier-liveness.mjs --self-test exit=0
node scripts/check-skill-identifier-liveness.mjs exit=0
Leg 1: 500 citations / 48 published files / 91,177 tokens / 5 exemptions
Leg 2: 9 registered exhaustive sections, 1 ledgered gap

Full derived family union from node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (33 families — all run, both invocation spellings), every one exit 0: entry-guard · parse-guard · watch-hint-literal · ratchet-remedy-authority · skill-identifier-liveness · agent-test-spelling · cli-command-ids · bash32-floor · pnpm-filter-targets · node-version · pnpm-acquisition · nul-bytes · cross-package-test-inputs · required-contexts · shard-attestation · stall-guard-budget · turbo-task-graph · workflow-status-functions · type-check-coverage · type-check-debt · dual-build-cjs-loads · pm-dispatch-gates · bare-root-worklist --self-test · aggregator-roster · ci-filter-parity · position-name-fold-loaders · self-test-wired · self-test-workflow-commands · step-collectors · whole-set-label-write · ci-failure --self-test · docs-audit/check-drift-comment.

Both convention-triggered obligations for adding a gate script were run and are green. check:type-check-debt was run after building the closure (turbo run build, 70/70 successful), because its --re-measure half silently measures a different world without it: OK — 29 ledger entries re-measured, 1531 raw tsc errors, none above its recorded number.

Repo-wide pnpm lint (eslint . --no-inline-config) run in full: exit 0 in 59s. No narrowing claimed.

node scripts/check-test-completeness.mjs returns exit 3 = PREREQUISITE NOT MET (it grades a saved turbo run test log that only CI produces). Recorded as NOT MEASURED — it is not a finding and not a failure.

The two edited gate scripts (check-ratchet-remedy-authority.mjs, bare-root-worklist.mjs) carry no vitest suite — repo-wide, no test file names either — so their --self-test IS their suite, and both were run green.

Changeset

None, and skip-changeset applied. Verified rather than assumed: the diff is scripts/**, .github/workflows/lint.yml, and one alias line in the private root manifest (@objectstack/spec-monorepo, private: true). Nothing under packages/, apps/ or examples/ changed, so this releases nothing — the workflow's own words for this case are "this PR edits a CI-internal script is the textbook skip-changeset case".

Scope

scripts/ and .github/workflows/lint.yml are not governed surfaces — regular code lane, opened as draft for the PM. Adjacent cards #13859 and #13835 are deliberately untouched. #13859 is adjacent but not subsumed: it is about a retirement-pin scanning one markdown file, a different instrument over a different question — this gate would not have caught it, because a withdrawn claim's sentence is semantics, and Leg 1 catches existence only.


Generated by Claude Code

…ills catalog
Leg 1 (phantom row): a skills/** table row citing an identifier that greps to
zero in packages/** is a red. Leg 2 (missing row): a section REGISTERED as
exhaustive over a schema symbol that does not document every member is a red.
One corpus walk, two legs.
Leg 2 is table-driven rather than inferred, and that is a measurement result:
auto-binding a doc table to any enum it is a subset of scores 1 true positive
in 3 on this corpus. The heuristic ships as --suggest, which cannot fail.
Zero skills/** edits: the five surviving Leg 1 sites and the one measured Leg 2
gap are ledgered, not fixed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT
- check-ratchet-remedy-authority CONTROL row (the sweep classified it `marked`
before the row was written; the row is recorded from that verdict).
- bare-root-worklist: REFUSE-WIDE for IMPL_ROOTS `packages` — the word index
reads 5405 of 5640 tracked files (96%), so the population IS the root and a
true declaration would name this gate on every card touching any package.
- The watch-hint self-test asserts the spelling without writing a second
declaration site; spelling it as a regex literal made the real one
unjudgeable ("2 declaration sites").
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT
@github-actionsgithub-actionsBot added size/xl ci/cd dependencies Pull requests that update a dependency file labels Aug 31, 2026
@huangyiirenehuangyiirene added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 31, 2026 — with Claude
@huangyiirene
huangyiirene marked this pull request as ready for review August 31, 2026 16:41
@huangyiirene
huangyiirene added this pull request to the merge queueAug 31, 2026
Merged via the queue into main with commit ea0708cAug 31, 2026
37 checks passed
@huangyiirene
huangyiirene deleted the claude/issue-13678-skill-identifier-liveness-gate branch August 31, 2026 17:05
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cddependenciesPull requests that update a dependency filesize/xlskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants

@huangyiirene@claude
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Two-legged identifier-liveness gate over the published skills catalog (#13678) by huangyiirene · Pull Request #13892 · objectstack-ai/objectstack · GitHub
Skip to content

Two-legged identifier-liveness gate over the published skills catalog (#13678) - #13892

Merged
huangyiirene merged 2 commits into
mainfrom
claude/issue-13678-skill-identifier-liveness-gate
Aug 31, 2026
Merged

Two-legged identifier-liveness gate over the published skills catalog (#13678)#13892
huangyiirene merged 2 commits into
mainfrom
claude/issue-13678-skill-identifier-liveness-gate

Conversation

@huangyiirene

Copy link
Copy Markdown
Collaborator

Fixes#13678

A two-legged identifier-liveness gate over the published skills/** catalog, plus its self-test and its registrations with the farm's convention gates. Zero skills/** edits — the gate reads that corpus, it does not fix it.

The two predicates, stated precisely

Leg 1 — phantom row. A CITATION is a markdown table row, outside a fenced block, whose FIRST cell is exactly one backticked token of identifier shape (dotted segments, optional [] as in bands[].key), optionally followed by a short parenthetical annotation. It is LIVE when every dotted segment appears as a word token in the IMPLEMENTATION INDEX. Zero hits on any segment is a red naming file, line, identifier and the failing segment.

The index is built from packages/** source only — never from skills/** and never from content/docs/**. That exclusion is the load-bearing line in the leg: an index containing the teaching corpus would find every citation in the corpus that cites it, so every row would be live, the finding set would be empty on any tree, and the gate would be a green light wired to nothing. dist/ is skipped for the mirror-image reason — a stale build output keeps a retired identifier alive for as long as nobody cleans it, making the verdict a function of the developer's working directory. Both are --self-test assertions, not prose.

Three deliberate refusals in the extractor, each measured rather than assumed:

  • The trailing annotation.`none` (default) is a citation of none. A first cut required the cell to be exactly a backticked token; that omission silently dropped every annotated row, and on the Leg 2 side it manufactured two false reds by making complete tables read as 4-of-5 and 3-of-4.
  • The negative column. A migration table's left column cites the dead spelling BY DESIGN (| Legacy | CEL |, | Old | New |). Reddening those runs the gate backwards. Columns whose HEADER names the pre-migration or negative side are structurally out of the population — structurally, not by ledger entry, so those files never enter a ledger whose budget a later edit could spend.
  • The fence. Code fences are skipped. This is the opposite choice from check-corpus-claim-drift, deliberately: that gate exists because its defect lived in a comment inside a fence. This one reads table rows, and a pipe inside a fenced example is not one.

Leg 2 — missing row. A BINDING is a human assertion, in a table in the gate, that one named section of one file is exhaustive over one named schema symbol. For each binding: resolve the symbol from its declared source (z.enum or a named string-literal union, comments stripped first), locate the heading, take the section down to the next heading of the same-or-higher level, and require every member to appear inside some backticked span in that section. A missing member is a red.

Two structural safety rules: the anchor must be unique in its file (an ambiguous heading fails rather than silently first-matching a different table than the row's author read), and the symbol must resolve (a binding naming a deleted enum is STALE and fails — without that, deleting the schema would turn the row green, the exact inversion of what the row is for).

Scope is the SECTION, not the file: an exhaustiveness claim covers what a reader sees under that heading, and a file-wide pool credits a member mentioned 800 lines away. Measured, the difference is real — NavItemVariant reads 7-of-9 against its section and 8-of-9 against its whole file.

Why Leg 2 is registered rather than inferred — the false-red measurements

The scope note asks for a conservative "presented as exhaustive" predicate. Three candidates were built and measured on the real corpus before this shape was chosen.

PredicateRecallPrecisionVerdict
(a) auto-bind to any enum the table is a subset of17 bindings1 true positive in 3cannot fail CI
(b) keyword exhaustiveness markers ("all", "every", "the N types")18 of 155 tablespoor before member comparisonunreliable claim detector
(c) table names its own schema symbol in backticks1 of 155 tables, and that one hedgedhighno recall, and zero skills/** edits permitted

(a) in detail. 17 bindings, 9 candidate reds on the first cut. Six were artefacts: two from the trailing-annotation bug above (MetadataLockSchema read 3-of-4 and SeedMode 4-of-5 while both tables were complete), one from an enumeration split across several tables in one file (FlowNodeAction, whose own lead says "20 built-in node types"), and three from prose naming its own partiality. Of the three that survived every fix, two are one enumAggregationMetricType carries six aggregation functions plus three number/string/boolean members whose comments read "Custom SQL expression returning X". A table titled "Basic Aggregation Functions" listing exactly the six is CORRECT, and this predicate calls it false. In two files.

(b) in detail. The top-ranked hit is skills/objectstack-ai/SKILL.md "Three action types dispatch headlessly" — it matches on both "every" and a numeral and is a deliberate 3-of-6 subset. Most other matches belong to a neighbouring sentence, not the table ("every author must remember to re-attach", "all optional").

So the table-driven shape is a measurement result, not a preference, and it follows the idiom PR #13742 established next door: the engine takes the table as a parameter and --self-test drives synthetic rows through it, so genericity is proven without filling the table. The heuristic is not discarded — it ships as --suggest, a non-failing discovery mode that lists unregistered candidates, so the registry has a feeder and the numbers above stay reproducible instead of living only in a comment.

False-red analysis on the current corpus — every hit dispositioned

Leg 1: 500 citations over 48 published files, checked against 91,177 word tokens from 5,405 files. Five rows survive the predicate. None is a phantom, and they are two structural classes:

SiteIdentifierKindWhy it cannot be indexed here
objectstack-pm-dispatch/SKILL.md:98,102,103backlogRepo, conventionsFile, routingLabelPrefixskill-owned-configKeys of a config file the CUSTOMER writes, consumed by the skill's own prose. The skill IS the implementation; there is no repo symbol to grep.
objectstack-ui/SKILL.md:557,562dayStart, showMidnightsibling-repoLive in ../objectui at packages/plugin-gantt/src/GanttView.tsx. This repo ships backend only, packages/console/dist is gitignored, and sdui.manifest.json does not carry gantt plugin props — checked, not assumed.

A sixth, bare_field in objectstack-formula, sits under a | Legacy | CEL | header and is removed structurally by the negative-column rule rather than ledgered.

Leg 2: 9 registered bindings. Eight are complete today and are now pinned against future drift: ApiMethod, ObjectAccessScopeSchema, HookEvent (x2 files), LifecycleClassSchema, SeedMode, ReportType, ActionType. The ninth is a real, known gap, ledgered rather than fixed: ### Navigation Item Types documents 7 of the 9 NavItemVariant members — action and component have no row. Filed as #13890, a sub-issue of the sweep program.

Registering ActionType against the UI skill's complete table while the AI skill documents a deliberate 3-of-6 subset of the same enum is the registry earning its keep: a binding names a SECTION, so the correct table is pinned and the subset is left alone.

One finding worth a reviewer's eye, which the gate deliberately does not act on.ApproverType has ten members including role, and the doc documents nine. Documenting the tenth would trip check:role-word, which makes "role" a reserved-forbidden word across this same corpus. Two gates over one corpus can be put in direct conflict by a schema member's name. No binding is registered for it, and that is a decision rather than an oversight.

Robustness to the pending sweep merges

The brief asks that in-flight sweeps not redden this. Checked, not assumed: #13833 touches objectstack-i18n, #13861 touches objectstack-upgrade and adds skills/spec-property-retirement. Neither touches any file this gate ledgers. Predicates are identifier-level throughout — there is no count pin anywhere in either leg, so a sweep that adds or rewrites prose moves nothing unless it adds a citation that greps to zero, which is the gate working.

The two ledgers move in OPPOSITE directions, deliberately

  • Leg 1 exemptions — PRUNE-ONLY.kind and note are human judgements about why an identifier is unreachable from this repo; nothing in the tree encodes them, so an --update that rewrote this list would invent classifications it cannot know. --update only DELETES entries the scan no longer reaches. Shrink-only by construction, not by policy: there is no code path that grows it.
  • Leg 2 gaps — REWRITTEN FROM THE TREE, shrink-only in spirit. A gap is a measured count and is fully derivable, so --update moves it whichever way the tree moved. Only policy tells those apart, so the new-gap and grown-gap messages mark that path ⛔ MAINTAINER-ONLY while the author's own remedy is offered first and unmarked.

A gap that SHRANK fails and asks for --update — an improvement that does not move the ledger leaves budget behind for a later edit to spend silently.

Ablation — the gate CAN fail, on both legs

Commit-before-mutate: the implementation was committed first, so the restore leg points at HEAD and HEAD really holds it. Each leg proves its mutation reached DISK before any verdict is read (an editor's exit code proves nothing — sed/perl -i/replace all exit 0 on zero hits), and each restore is proven by blob-hash equality against the HEAD blob, never by an exit code. The script carries a trap ... EXIT INT TERM, restores with git checkout HEAD -- PATH (never the bare form, which restores from the index and would hand the mutation straight back at exit 0), and treats an empty hash as FAILURE rather than "nothing to compare".

=== BASELINE (unmutated tree) ===
exit=0
restored OK skills/objectstack-ui/SKILL.md blob=53121751928a… == HEAD:…
restored OK skills/objectstack-api/SKILL.md blob=c73e66997112… == HEAD:…
=== LEG 1 ABLATION: inject a PHANTOM ROW ===
injected-token count: 1 blob 53121751928a… -> c134f11ea385…
GATE exit=1
[leg1-phantom] skills/objectstack-ui/SKILL.md:642 cites `zzPhantomKeyNeverImplemented`
and the segment `zzPhantomKeyNeverImplemented` has ZERO hits in packages.
restored OK blob=53121751928a… == HEAD gate after restore exit=0
=== LEG 2 ABLATION: delete a documented MEMBER row ===
removed-row count now: 0 blob c73e66997112… -> 11673302ee99…
GATE exit=1
[leg2-missing-row] skills/objectstack-api/SKILL.md — ## API Methods (Operations) is
registered exhaustive over `ApiMethod` (packages/spec/src/data/object.zod.ts) and
does not document `bulk`.
restored OK blob=c73e66997112… == HEAD gate after restore exit=0

git status --porcelain is empty on the tree this PR was pushed from.

Registrations

  • check-ratchet-remedy-authority — a CONTROL row. The sweep classified this gate markedbefore the row was written, and the row is recorded from that verdict rather than from author intent; the corpus is set-equality audited both ways, so the row was demanded, not volunteered. Getting there needed the remedy text to carry real shrink testimony — the first wording said "weakens the gate", which the detector cannot anchor. Farm now: 12 marked, 6 refusing, 27 control rows.
  • bare-root-worklistREFUSE-WIDE for IMPL_ROOTS packages. The index reads 5,405 of 5,640 tracked files under that root (96%), so the population is not a subset of the root, it IS the root; a true declaration would name this gate on every card touching any package. Its other root, skills, IS declared beside the constant via ROOT_DIR_WATCH_HINTS — small, and the surface the gate exists for.
  • check-watch-hint-literal — the watch-hint self-test asserts the spelling without writing a second declaration site. Spelling the whole const NAME = [...] as a regex literal creates one, and the gate then reported "2 declaration sites — this gate cannot judge a declaration it cannot locate". Measured, then fixed.
  • lint.yml — one step in the job that carries the whole check:* family, with no paths filter. That matters more here than for its neighbours: the gate's two inputs are skills/** and packages/**, so a filter on either would let the OTHER side drift a member in or an identifier out without ever running the check that notices.

Verification

All readings taken on 1d84cf3b4, a clean tree, exit codes captured before any pipe.

node scripts/check-skill-identifier-liveness.mjs --self-test exit=0
node scripts/check-skill-identifier-liveness.mjs exit=0
Leg 1: 500 citations / 48 published files / 91,177 tokens / 5 exemptions
Leg 2: 9 registered exhaustive sections, 1 ledgered gap

Full derived family union from node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (33 families — all run, both invocation spellings), every one exit 0: entry-guard · parse-guard · watch-hint-literal · ratchet-remedy-authority · skill-identifier-liveness · agent-test-spelling · cli-command-ids · bash32-floor · pnpm-filter-targets · node-version · pnpm-acquisition · nul-bytes · cross-package-test-inputs · required-contexts · shard-attestation · stall-guard-budget · turbo-task-graph · workflow-status-functions · type-check-coverage · type-check-debt · dual-build-cjs-loads · pm-dispatch-gates · bare-root-worklist --self-test · aggregator-roster · ci-filter-parity · position-name-fold-loaders · self-test-wired · self-test-workflow-commands · step-collectors · whole-set-label-write · ci-failure --self-test · docs-audit/check-drift-comment.

Both convention-triggered obligations for adding a gate script were run and are green. check:type-check-debt was run after building the closure (turbo run build, 70/70 successful), because its --re-measure half silently measures a different world without it: OK — 29 ledger entries re-measured, 1531 raw tsc errors, none above its recorded number.

Repo-wide pnpm lint (eslint . --no-inline-config) run in full: exit 0 in 59s. No narrowing claimed.

node scripts/check-test-completeness.mjs returns exit 3 = PREREQUISITE NOT MET (it grades a saved turbo run test log that only CI produces). Recorded as NOT MEASURED — it is not a finding and not a failure.

The two edited gate scripts (check-ratchet-remedy-authority.mjs, bare-root-worklist.mjs) carry no vitest suite — repo-wide, no test file names either — so their --self-test IS their suite, and both were run green.

Changeset

None, and skip-changeset applied. Verified rather than assumed: the diff is scripts/**, .github/workflows/lint.yml, and one alias line in the private root manifest (@objectstack/spec-monorepo, private: true). Nothing under packages/, apps/ or examples/ changed, so this releases nothing — the workflow's own words for this case are "this PR edits a CI-internal script is the textbook skip-changeset case".

Scope

scripts/ and .github/workflows/lint.yml are not governed surfaces — regular code lane, opened as draft for the PM. Adjacent cards #13859 and #13835 are deliberately untouched. #13859 is adjacent but not subsumed: it is about a retirement-pin scanning one markdown file, a different instrument over a different question — this gate would not have caught it, because a withdrawn claim's sentence is semantics, and Leg 1 catches existence only.


Generated by Claude Code

…ills catalog
Leg 1 (phantom row): a skills/** table row citing an identifier that greps to
zero in packages/** is a red. Leg 2 (missing row): a section REGISTERED as
exhaustive over a schema symbol that does not document every member is a red.
One corpus walk, two legs.
Leg 2 is table-driven rather than inferred, and that is a measurement result:
auto-binding a doc table to any enum it is a subset of scores 1 true positive
in 3 on this corpus. The heuristic ships as --suggest, which cannot fail.
Zero skills/** edits: the five surviving Leg 1 sites and the one measured Leg 2
gap are ledgered, not fixed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT
- check-ratchet-remedy-authority CONTROL row (the sweep classified it `marked`
before the row was written; the row is recorded from that verdict).
- bare-root-worklist: REFUSE-WIDE for IMPL_ROOTS `packages` — the word index
reads 5405 of 5640 tracked files (96%), so the population IS the root and a
true declaration would name this gate on every card touching any package.
- The watch-hint self-test asserts the spelling without writing a second
declaration site; spelling it as a regex literal made the real one
unjudgeable ("2 declaration sites").
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT
@github-actionsgithub-actionsBot added size/xl ci/cd dependencies Pull requests that update a dependency file labels Aug 31, 2026
@huangyiirenehuangyiirene added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 31, 2026 — with Claude
@huangyiirene
huangyiirene marked this pull request as ready for review August 31, 2026 16:41
@huangyiirene
huangyiirene added this pull request to the merge queueAug 31, 2026
Merged via the queue into main with commit ea0708cAug 31, 2026
37 checks passed
@huangyiirene
huangyiirene deleted the claude/issue-13678-skill-identifier-liveness-gate branch August 31, 2026 17:05
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cddependenciesPull requests that update a dependency filesize/xlskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants

@huangyiirene@claude
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); })(); Two-legged identifier-liveness gate over the published skills catalog (#13678) by huangyiirene · Pull Request #13892 · objectstack-ai/objectstack · GitHub
Skip to content

Two-legged identifier-liveness gate over the published skills catalog (#13678) - #13892

Merged
huangyiirene merged 2 commits into
mainfrom
claude/issue-13678-skill-identifier-liveness-gate
Aug 31, 2026
Merged

Two-legged identifier-liveness gate over the published skills catalog (#13678)#13892
huangyiirene merged 2 commits into
mainfrom
claude/issue-13678-skill-identifier-liveness-gate

Conversation

@huangyiirene

Copy link
Copy Markdown
Collaborator

Fixes#13678

A two-legged identifier-liveness gate over the published skills/** catalog, plus its self-test and its registrations with the farm's convention gates. Zero skills/** edits — the gate reads that corpus, it does not fix it.

The two predicates, stated precisely

Leg 1 — phantom row. A CITATION is a markdown table row, outside a fenced block, whose FIRST cell is exactly one backticked token of identifier shape (dotted segments, optional [] as in bands[].key), optionally followed by a short parenthetical annotation. It is LIVE when every dotted segment appears as a word token in the IMPLEMENTATION INDEX. Zero hits on any segment is a red naming file, line, identifier and the failing segment.

The index is built from packages/** source only — never from skills/** and never from content/docs/**. That exclusion is the load-bearing line in the leg: an index containing the teaching corpus would find every citation in the corpus that cites it, so every row would be live, the finding set would be empty on any tree, and the gate would be a green light wired to nothing. dist/ is skipped for the mirror-image reason — a stale build output keeps a retired identifier alive for as long as nobody cleans it, making the verdict a function of the developer's working directory. Both are --self-test assertions, not prose.

Three deliberate refusals in the extractor, each measured rather than assumed:

  • The trailing annotation.`none` (default) is a citation of none. A first cut required the cell to be exactly a backticked token; that omission silently dropped every annotated row, and on the Leg 2 side it manufactured two false reds by making complete tables read as 4-of-5 and 3-of-4.
  • The negative column. A migration table's left column cites the dead spelling BY DESIGN (| Legacy | CEL |, | Old | New |). Reddening those runs the gate backwards. Columns whose HEADER names the pre-migration or negative side are structurally out of the population — structurally, not by ledger entry, so those files never enter a ledger whose budget a later edit could spend.
  • The fence. Code fences are skipped. This is the opposite choice from check-corpus-claim-drift, deliberately: that gate exists because its defect lived in a comment inside a fence. This one reads table rows, and a pipe inside a fenced example is not one.

Leg 2 — missing row. A BINDING is a human assertion, in a table in the gate, that one named section of one file is exhaustive over one named schema symbol. For each binding: resolve the symbol from its declared source (z.enum or a named string-literal union, comments stripped first), locate the heading, take the section down to the next heading of the same-or-higher level, and require every member to appear inside some backticked span in that section. A missing member is a red.

Two structural safety rules: the anchor must be unique in its file (an ambiguous heading fails rather than silently first-matching a different table than the row's author read), and the symbol must resolve (a binding naming a deleted enum is STALE and fails — without that, deleting the schema would turn the row green, the exact inversion of what the row is for).

Scope is the SECTION, not the file: an exhaustiveness claim covers what a reader sees under that heading, and a file-wide pool credits a member mentioned 800 lines away. Measured, the difference is real — NavItemVariant reads 7-of-9 against its section and 8-of-9 against its whole file.

Why Leg 2 is registered rather than inferred — the false-red measurements

The scope note asks for a conservative "presented as exhaustive" predicate. Three candidates were built and measured on the real corpus before this shape was chosen.

PredicateRecallPrecisionVerdict
(a) auto-bind to any enum the table is a subset of17 bindings1 true positive in 3cannot fail CI
(b) keyword exhaustiveness markers ("all", "every", "the N types")18 of 155 tablespoor before member comparisonunreliable claim detector
(c) table names its own schema symbol in backticks1 of 155 tables, and that one hedgedhighno recall, and zero skills/** edits permitted

(a) in detail. 17 bindings, 9 candidate reds on the first cut. Six were artefacts: two from the trailing-annotation bug above (MetadataLockSchema read 3-of-4 and SeedMode 4-of-5 while both tables were complete), one from an enumeration split across several tables in one file (FlowNodeAction, whose own lead says "20 built-in node types"), and three from prose naming its own partiality. Of the three that survived every fix, two are one enumAggregationMetricType carries six aggregation functions plus three number/string/boolean members whose comments read "Custom SQL expression returning X". A table titled "Basic Aggregation Functions" listing exactly the six is CORRECT, and this predicate calls it false. In two files.

(b) in detail. The top-ranked hit is skills/objectstack-ai/SKILL.md "Three action types dispatch headlessly" — it matches on both "every" and a numeral and is a deliberate 3-of-6 subset. Most other matches belong to a neighbouring sentence, not the table ("every author must remember to re-attach", "all optional").

So the table-driven shape is a measurement result, not a preference, and it follows the idiom PR #13742 established next door: the engine takes the table as a parameter and --self-test drives synthetic rows through it, so genericity is proven without filling the table. The heuristic is not discarded — it ships as --suggest, a non-failing discovery mode that lists unregistered candidates, so the registry has a feeder and the numbers above stay reproducible instead of living only in a comment.

False-red analysis on the current corpus — every hit dispositioned

Leg 1: 500 citations over 48 published files, checked against 91,177 word tokens from 5,405 files. Five rows survive the predicate. None is a phantom, and they are two structural classes:

SiteIdentifierKindWhy it cannot be indexed here
objectstack-pm-dispatch/SKILL.md:98,102,103backlogRepo, conventionsFile, routingLabelPrefixskill-owned-configKeys of a config file the CUSTOMER writes, consumed by the skill's own prose. The skill IS the implementation; there is no repo symbol to grep.
objectstack-ui/SKILL.md:557,562dayStart, showMidnightsibling-repoLive in ../objectui at packages/plugin-gantt/src/GanttView.tsx. This repo ships backend only, packages/console/dist is gitignored, and sdui.manifest.json does not carry gantt plugin props — checked, not assumed.

A sixth, bare_field in objectstack-formula, sits under a | Legacy | CEL | header and is removed structurally by the negative-column rule rather than ledgered.

Leg 2: 9 registered bindings. Eight are complete today and are now pinned against future drift: ApiMethod, ObjectAccessScopeSchema, HookEvent (x2 files), LifecycleClassSchema, SeedMode, ReportType, ActionType. The ninth is a real, known gap, ledgered rather than fixed: ### Navigation Item Types documents 7 of the 9 NavItemVariant members — action and component have no row. Filed as #13890, a sub-issue of the sweep program.

Registering ActionType against the UI skill's complete table while the AI skill documents a deliberate 3-of-6 subset of the same enum is the registry earning its keep: a binding names a SECTION, so the correct table is pinned and the subset is left alone.

One finding worth a reviewer's eye, which the gate deliberately does not act on.ApproverType has ten members including role, and the doc documents nine. Documenting the tenth would trip check:role-word, which makes "role" a reserved-forbidden word across this same corpus. Two gates over one corpus can be put in direct conflict by a schema member's name. No binding is registered for it, and that is a decision rather than an oversight.

Robustness to the pending sweep merges

The brief asks that in-flight sweeps not redden this. Checked, not assumed: #13833 touches objectstack-i18n, #13861 touches objectstack-upgrade and adds skills/spec-property-retirement. Neither touches any file this gate ledgers. Predicates are identifier-level throughout — there is no count pin anywhere in either leg, so a sweep that adds or rewrites prose moves nothing unless it adds a citation that greps to zero, which is the gate working.

The two ledgers move in OPPOSITE directions, deliberately

  • Leg 1 exemptions — PRUNE-ONLY.kind and note are human judgements about why an identifier is unreachable from this repo; nothing in the tree encodes them, so an --update that rewrote this list would invent classifications it cannot know. --update only DELETES entries the scan no longer reaches. Shrink-only by construction, not by policy: there is no code path that grows it.
  • Leg 2 gaps — REWRITTEN FROM THE TREE, shrink-only in spirit. A gap is a measured count and is fully derivable, so --update moves it whichever way the tree moved. Only policy tells those apart, so the new-gap and grown-gap messages mark that path ⛔ MAINTAINER-ONLY while the author's own remedy is offered first and unmarked.

A gap that SHRANK fails and asks for --update — an improvement that does not move the ledger leaves budget behind for a later edit to spend silently.

Ablation — the gate CAN fail, on both legs

Commit-before-mutate: the implementation was committed first, so the restore leg points at HEAD and HEAD really holds it. Each leg proves its mutation reached DISK before any verdict is read (an editor's exit code proves nothing — sed/perl -i/replace all exit 0 on zero hits), and each restore is proven by blob-hash equality against the HEAD blob, never by an exit code. The script carries a trap ... EXIT INT TERM, restores with git checkout HEAD -- PATH (never the bare form, which restores from the index and would hand the mutation straight back at exit 0), and treats an empty hash as FAILURE rather than "nothing to compare".

=== BASELINE (unmutated tree) ===
exit=0
restored OK skills/objectstack-ui/SKILL.md blob=53121751928a… == HEAD:…
restored OK skills/objectstack-api/SKILL.md blob=c73e66997112… == HEAD:…
=== LEG 1 ABLATION: inject a PHANTOM ROW ===
injected-token count: 1 blob 53121751928a… -> c134f11ea385…
GATE exit=1
[leg1-phantom] skills/objectstack-ui/SKILL.md:642 cites `zzPhantomKeyNeverImplemented`
and the segment `zzPhantomKeyNeverImplemented` has ZERO hits in packages.
restored OK blob=53121751928a… == HEAD gate after restore exit=0
=== LEG 2 ABLATION: delete a documented MEMBER row ===
removed-row count now: 0 blob c73e66997112… -> 11673302ee99…
GATE exit=1
[leg2-missing-row] skills/objectstack-api/SKILL.md — ## API Methods (Operations) is
registered exhaustive over `ApiMethod` (packages/spec/src/data/object.zod.ts) and
does not document `bulk`.
restored OK blob=c73e66997112… == HEAD gate after restore exit=0

git status --porcelain is empty on the tree this PR was pushed from.

Registrations

  • check-ratchet-remedy-authority — a CONTROL row. The sweep classified this gate markedbefore the row was written, and the row is recorded from that verdict rather than from author intent; the corpus is set-equality audited both ways, so the row was demanded, not volunteered. Getting there needed the remedy text to carry real shrink testimony — the first wording said "weakens the gate", which the detector cannot anchor. Farm now: 12 marked, 6 refusing, 27 control rows.
  • bare-root-worklistREFUSE-WIDE for IMPL_ROOTS packages. The index reads 5,405 of 5,640 tracked files under that root (96%), so the population is not a subset of the root, it IS the root; a true declaration would name this gate on every card touching any package. Its other root, skills, IS declared beside the constant via ROOT_DIR_WATCH_HINTS — small, and the surface the gate exists for.
  • check-watch-hint-literal — the watch-hint self-test asserts the spelling without writing a second declaration site. Spelling the whole const NAME = [...] as a regex literal creates one, and the gate then reported "2 declaration sites — this gate cannot judge a declaration it cannot locate". Measured, then fixed.
  • lint.yml — one step in the job that carries the whole check:* family, with no paths filter. That matters more here than for its neighbours: the gate's two inputs are skills/** and packages/**, so a filter on either would let the OTHER side drift a member in or an identifier out without ever running the check that notices.

Verification

All readings taken on 1d84cf3b4, a clean tree, exit codes captured before any pipe.

node scripts/check-skill-identifier-liveness.mjs --self-test exit=0
node scripts/check-skill-identifier-liveness.mjs exit=0
Leg 1: 500 citations / 48 published files / 91,177 tokens / 5 exemptions
Leg 2: 9 registered exhaustive sections, 1 ledgered gap

Full derived family union from node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (33 families — all run, both invocation spellings), every one exit 0: entry-guard · parse-guard · watch-hint-literal · ratchet-remedy-authority · skill-identifier-liveness · agent-test-spelling · cli-command-ids · bash32-floor · pnpm-filter-targets · node-version · pnpm-acquisition · nul-bytes · cross-package-test-inputs · required-contexts · shard-attestation · stall-guard-budget · turbo-task-graph · workflow-status-functions · type-check-coverage · type-check-debt · dual-build-cjs-loads · pm-dispatch-gates · bare-root-worklist --self-test · aggregator-roster · ci-filter-parity · position-name-fold-loaders · self-test-wired · self-test-workflow-commands · step-collectors · whole-set-label-write · ci-failure --self-test · docs-audit/check-drift-comment.

Both convention-triggered obligations for adding a gate script were run and are green. check:type-check-debt was run after building the closure (turbo run build, 70/70 successful), because its --re-measure half silently measures a different world without it: OK — 29 ledger entries re-measured, 1531 raw tsc errors, none above its recorded number.

Repo-wide pnpm lint (eslint . --no-inline-config) run in full: exit 0 in 59s. No narrowing claimed.

node scripts/check-test-completeness.mjs returns exit 3 = PREREQUISITE NOT MET (it grades a saved turbo run test log that only CI produces). Recorded as NOT MEASURED — it is not a finding and not a failure.

The two edited gate scripts (check-ratchet-remedy-authority.mjs, bare-root-worklist.mjs) carry no vitest suite — repo-wide, no test file names either — so their --self-test IS their suite, and both were run green.

Changeset

None, and skip-changeset applied. Verified rather than assumed: the diff is scripts/**, .github/workflows/lint.yml, and one alias line in the private root manifest (@objectstack/spec-monorepo, private: true). Nothing under packages/, apps/ or examples/ changed, so this releases nothing — the workflow's own words for this case are "this PR edits a CI-internal script is the textbook skip-changeset case".

Scope

scripts/ and .github/workflows/lint.yml are not governed surfaces — regular code lane, opened as draft for the PM. Adjacent cards #13859 and #13835 are deliberately untouched. #13859 is adjacent but not subsumed: it is about a retirement-pin scanning one markdown file, a different instrument over a different question — this gate would not have caught it, because a withdrawn claim's sentence is semantics, and Leg 1 catches existence only.


Generated by Claude Code

…ills catalog
Leg 1 (phantom row): a skills/** table row citing an identifier that greps to
zero in packages/** is a red. Leg 2 (missing row): a section REGISTERED as
exhaustive over a schema symbol that does not document every member is a red.
One corpus walk, two legs.
Leg 2 is table-driven rather than inferred, and that is a measurement result:
auto-binding a doc table to any enum it is a subset of scores 1 true positive
in 3 on this corpus. The heuristic ships as --suggest, which cannot fail.
Zero skills/** edits: the five surviving Leg 1 sites and the one measured Leg 2
gap are ledgered, not fixed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT
- check-ratchet-remedy-authority CONTROL row (the sweep classified it `marked`
before the row was written; the row is recorded from that verdict).
- bare-root-worklist: REFUSE-WIDE for IMPL_ROOTS `packages` — the word index
reads 5405 of 5640 tracked files (96%), so the population IS the root and a
true declaration would name this gate on every card touching any package.
- The watch-hint self-test asserts the spelling without writing a second
declaration site; spelling it as a regex literal made the real one
unjudgeable ("2 declaration sites").
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnE7G31tqbxN1rqpQmzurT
@github-actionsgithub-actionsBot added size/xl ci/cd dependencies Pull requests that update a dependency file labels Aug 31, 2026
@huangyiirenehuangyiirene added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 31, 2026 — with Claude
@huangyiirene
huangyiirene marked this pull request as ready for review August 31, 2026 16:41
@huangyiirene
huangyiirene added this pull request to the merge queueAug 31, 2026
Merged via the queue into main with commit ea0708cAug 31, 2026
37 checks passed
@huangyiirene
huangyiirene deleted the claude/issue-13678-skill-identifier-liveness-gate branch August 31, 2026 17:05
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cddependenciesPull requests that update a dependency filesize/xlskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants

@huangyiirene@claude