Skip to content

Strip internal issue-id references from the published skill catalog, and gate their return - #11931

Merged
hotlong merged 2 commits into
mainfrom
claude/issue-11052-strip-internal-issue-ids
Aug 25, 2026
Merged

Strip internal issue-id references from the published skill catalog, and gate their return#11931
hotlong merged 2 commits into
mainfrom
claude/issue-11052-strip-internal-issue-ids

Conversation

@claude

@claudeclaudeBot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Fixes#11052

Implements the maintainer ruling of 2026-08-23 (option A, recorded in comment 5386683324): strip the internal issue-id references from the published skill catalog, add a gate against their return, and land the per-file ceiling drops in the same PR.

⚠️Governed surface (skills/** + the gate script) — draft, human merge. Not queued, not armed, not flipped ready.


1. The population, re-measured

The card's 92 was a 2026-08-22 count over SKILL.md files only. Re-measured on this branch's base 3f571a6d2, over the whole published tree:

bucketsitesdisposition
SKILL.md (11 files, hand-authored)90stripped
other hand-authored (rules/, references/, evals/, README.md)41stripped
generated artifacts (references/_index.md ×9, references/react-blocks.md)14out of scope — see below
total id-shaped tokens in skills/**145

Two corrections to the filing count, both load-bearing:

  • The card's 92 included 2 false positives.skills/objectstack-ui/SKILL.md carries #6366f1 and #4169E1 — CSS hex colours that a #[0-9]{3,5} scan reads as issue ids. The true SKILL.md population was 90. (A third, #3498db, sits in rules/field-types.md.) This is exactly the precision the gate had to get right, and those three colours are now pinned as must-not-fire cases.
  • The card's scan missed 41 sites outside SKILL.md, including 15 in objectstack-data/references/data-hooks.md — a hand-authored file, despite living under references/.
Per-file breakdown of the 131 stripped sites
17 objectstack-data/SKILL.md 6 objectstack-data/rules/indexing.md
15 objectstack-data/references/data-hooks.md 6 objectstack-ai/SKILL.md
14 objectstack-automation/SKILL.md 6 objectstack-api/SKILL.md
14 objectstack-formula/SKILL.md 5 objectstack-i18n/SKILL.md
13 objectstack-ui/SKILL.md 5 objectstack-query/rules/aggregation.md
8 objectstack-query/SKILL.md 4 objectstack-data/rules/hooks.md
3 objectstack-platform/SKILL.md 2 objectstack-data/rules/field-types.md
3 objectstack-pm-dispatch/SKILL.md 2 objectstack-data/rules/validation.md
2 objectstack-query/evals/README.md 2 objectstack-query/rules/pagination.md
2 objectstack-automation/evals/approvals/test-revise-loop.md
1 objectstack-upgrade/SKILL.md 1 README.md

Reachability — the ruling's fallback-C check

Zero of the 145 sites is a reachable reference. Every one is a bare #NNNN (or framework#3582); none is a URL. The only issue-URL-shaped string in the catalog is github.com/acme/hotcrm/pull/456, a fictional example in the PM skill.

One honest nuance for the review seat: objectstack-ai/objectstack is a public repository, so a determined reader could hand-construct an issue URL. That slightly softens the card's "cannot resolve one" wording — but it does not touch the ruling's basis. A bare #4286 inside a file in the customer's own project autolinks to nothing, names no repo, and is indistinguishable from a reference to the customer's own tracker. It remains a citation-shaped token that costs tokens and buys the paying reader nothing.

What was NOT stripped

14 sites in generated artifacts.references/_index.md (×9) and references/react-blocks.md carry Auto-generated — do not edit / GENERATED … do not edit headers; their ids are projected from .describe() / TSDoc in packages/spec. Fixing them is a spec-source edit plus a regeneration, which also rewrites content/docs/references/ and answers to different gates — an unmeasured widening riding on a measured card. Filed separately (see below) and exempted by path in the gate, with the reason recorded in the source.

Nothing else. The fallback-C candidate was ruled and is now stripped too — see the rework note below. The hand-authored published corpus contains zero concrete internal ids, with no exemption anywhere in the gate.

Rework round (maintainer ruling 2026-08-25 — option B)

The first cut kept two ids at skills/objectstack-pm-dispatch/SKILL.md:70 as the ruling's fallback C: /pm-dispatch #128 #131, the documented invocation grammar of the command that page teaches, held in a narrow per-passage allowlist and escalated for a per-passage ruling rather than kept silently.

Two things then happened, and they had the same fix:

  1. CI went red on check:ratchet-remedy-authority (required, "Type Check · source gates") with two findings. UNMARKED: the new rule's failure text handed the author a remedy that expands a registry ("…adding an EXAMPLE_ID_ALLOWLIST entry…") while neither marking that path ⛔ MAINTAINER-ONLY nor turning it down. UNCLASSIFIED: check-doc-authoring.mjs was reached by that gate's sweep and its control corpus did not cover it.
  2. The open question was ruled B.

Option B dissolves finding 1 at the root instead of marking it. The line now reads:

/pm-dispatch #<n> #<n> # two named issues, nothing else

— the same placeholder spelling the sibling "filed as #<n>: …" site in that file already uses (that one was rewritten this way in the first cut). The # still teaches the argument grammar; the numbers stop impersonating a citation. The edit is byte-neutral, so the pm-dispatch ceiling is unmoved at 14239.

EXAMPLE_ID_ALLOWLIST, its predicate, its call site, its four self-test pins and every mention in the failure text are deleted. The rule now offers no expansion path at all and says so; its verdict under the authority gate is excluded (no ratchet-expanding remedy on any path). Its two allowlist self-test cases are replaced by the pair that actually matters: the #<n> placeholder — the remedy the failure text prescribes — must PASS, and the concrete ids it replaced must stay RED. check-doc-authoring.mjs is now recorded in that gate's control corpus as excluded, with the reason.

The general shape is written into the source next to where the list used to be: a passage that seems to need an example id needs a placeholder instead. A growable allowlist would have been the one place a genuine citation could come to rest — "it's an example" is exactly what the author of the next one would believe.

2. The strip — teaching kept, citation dropped

All 131 in-scope sites cleared across 20 files (129 in the first cut, the last 2 in the rework round above). Where a customer-resolvable anchor existed, it replaces the id rather than leaving a gap:

beforeafter
> ✅ **Enforced since #4286.** The engine applies …> ✅ **Enforced.** The engine applies …
// ❌ cursor was removed in #4286 — the tombstone …// ❌ cursor was removed in protocol 17 — the tombstone …
⚠️ requiresConfirmation … REMOVED (#3715, ADR-0033 §2)⚠️ requiresConfirmation … REMOVED (ADR-0033 §2)
// The former tenantId alias was removed in #3290.// The former tenantId alias was removed in v16.
`onEmptyApprovers

No sentence lost its lesson; sentences that existed only to cite went entirely. Four line-wrap repairs were needed where a citation opened a wrapped line ((#5038).** A predicate … would otherwise have left an orphan .**), and three code/table alignment runs were restored after the mechanical pass collapsed them.

3. The gate — check-doc-authoring.mjs, second corpus rule

Bans bare internal issue ids on the customer-facing surface only. .claude/ and docs/ are out of scope by design: an internal reader has the tracker, git log and the ADRs, so an id there resolves. The ban follows the audience, not the file type.

It has its own walker, deliberately. The existing bare-literal walk honours SKIP_DIRS, which contains references — correct for that rule, wrong for this one, since a ninth of this rule's population lives in references/data-hooks.md. Reusing it produced a gate that ran, passed, and could not see the files it exists to guard; the self-test case that proves the walker reaches references/failed until the dedicated walker existed.

Precision. The detector, described in words because this page's byte sanitizer eats the exclamation mark out of a literal lookaround: a negative lookbehind rejecting a preceding # or ampersand, then a literal #, then 3–5 digits, then a negative lookahead rejecting any following alphanumeric. (The exact pattern is INTERNAL_ID_SOURCE in the gate.) That trailing lookahead is what carries the precision:

must not firewhy it doesn't
#6366f1, #4169E1, #3498db, #123456next char after the digit run is a word char
v17, 17.0.0, protocol 17no #
400 INVALID_FIELD, 404no #
fields[0], fields[12]no #
The #1 authoring mistakeone digit, below the 3-digit floor
### 4286 things# preceded by #
&#8212;# preceded by &

The cross-repo spelling framework#3582 (real, in objectstack-ui/SKILL.md) is caught — no leading-boundary lookbehind, on purpose.

Scanned population: 38 files (48 .md in skills/, minus the 9 _index.md and 1 react-blocks.md generated artifacts — every one verified to carry a generated marker). An empty scan is a hard error, matching the file's existing #4932 discipline.

Reverse verification (against the real gate, not the self-test)

Direction predicted before running; mutation proven on disk in both legs; restore under an EXIT INT TERM trap.

=== LEG 1: mutate ===
appended marker present BEFORE: 0 (want 0)
appended marker present AFTER : 1 (want 1)
skills/objectstack-query/SKILL.md | 2 ++
=== LEG 1: gate must be RED ===
RED_EXIT=1 (want non-zero)
✗ Internal issue-id reference(s) in the PUBLISHED skill catalog:
skills/objectstack-query/SKILL.md:675 #4286
=== LEG 2: restore ===
marker present after restore: 0 (want 0)
=== LEG 2: gate must be GREEN ===
GREEN_EXIT=0 (want 0)
✓ doc authoring guard: 38 published skill files clean — no internal issue-id references.
=== VERDICT ===
PASS — red on the planted id, green when restored, mutation proven on disk both ways.

The self-test additionally drives red/green for a planted id in prose, in a fenced code comment, and in the repo#NNNN spelling, plus all twelve precision cases and the placeholder pair (the #<n> form the failure text prescribes passes; the concrete ids it replaced are red, with nothing to add them to).

4. Ceiling drops (same PR, shrink-only)

Every ceiling re-measured to its post-strip reading:

skills/objectstack-ai/SKILL.md 6824 -> 6806 (-18)
skills/objectstack-api/SKILL.md 6342 -> 6331 (-11)
skills/objectstack-automation/SKILL.md 12543 -> 12511 (-32)
skills/objectstack-data/SKILL.md 13817 -> 13783 (-34)
skills/objectstack-formula/SKILL.md 6055 -> 6002 (-53)
skills/objectstack-i18n/SKILL.md 6349 -> 6338 (-11)
skills/objectstack-platform/SKILL.md 12716 -> 12705 (-11)
skills/objectstack-pm-dispatch/SKILL.md 14239 -> 14239 ( 0) byte-neutral edit
skills/objectstack-query/SKILL.md 5569 -> 5552 (-17)
skills/objectstack-ui/SKILL.md 25154 -> 25125 (-29)
skills/objectstack-upgrade/SKILL.md 8335 -> 8333 (-2)
bundle total 117943 -> 117725 (-218)

#10402's reserved headroom was already spent.objectstack-data measured 13817 against a 13817 ceiling on the base — exactly zero headroom, which is the ratchet header's "the headroom returns to zero on its own", observed. Lowering it takes nothing that PR was holding. Recorded as a third basis, CEILING_BASIS.strippedInternalIds, pinned in the self-test.

Some rows drop by more than their file shrank (formula shrank 34, ceiling drops 53) — a lowering also reclaims pre-existing slack, which is the shrink-only discipline working as intended.

Whole-package readings (2026-08-21 size ruling)

scopelinestokens
entire skills/ tree17848 → 17847 (−1)188183 → 187921 (−262)
the 11 ratcheted SKILL.md117943 → 117725 (−218)

Files not priced by the ratchet (rules/*.md, references/data-hooks.md, evals/**, README.md) have no ceiling to move — their before/after sit in the per-file table above, per the #11869 pattern. Every reading is a reduction; this PR adds no prose to the published bundle.

5. Gates

Derived with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (no path args) — now 15 matched families, all run, plus the four skills families. Exit codes captured before any pipe. Re-run at 9c710e47, the final commit, on a clean tree.

⚠️check:ratchet-remedy-authority was not derivable in the first round — it sweeps scripts/** but assembles that path (join(REPO_ROOT, 'scripts')), so hintCovers sees no literal and the family sits in the Undetermined (source names no path at all — NOT known irrelevant) bucket, 53 deep. It joins the matched list here only because this round edits its own script. That gap is filed as #11935.

gateverdict line
check:doc-authoring✓ 389 files clean — no bare metadata literals. / ✓ 38 published skill files clean — no internal issue-id references.
check-doc-authoring --self-test✓ … the published-catalog internal-id rule … all hold.
check:skills-token-ratchet✓ 11 published SKILL.md within their ceilings.
check-skills-token-ratchet --self-test✓ 35 cases pass.
check:skill-examples✅ 256 prose examples type-check across 3 surface(s)
check:skill-docs✅ Skill docs in sync
check:skill-refs✅ 9 generated files in sync with packages/spec
check:skill-frame-syncexit 0
check:skill-compatibilityexit 0
check:role-wordexit 0
check:nul-bytesexit 0
check:agent-test-spellingexit 0
check:cross-package-test-inputsexit 0
check:doc-formula-expressionsexit 0
check:entry-guard · check:parse-guard · check:pnpm-filter-targetsexit 0
check:pm-governed-merges✓ check-governed-merges --self-test: 129 assertions
check:ratchet-remedy-authorityOK check-ratchet-remedy-authority: 138 scripts swept … 125 hand out no ratchet-expanding remedy. Control corpus: 23 hand-classified scripts, set-equality audited both ways.
check-ci-filter-parity · check-cross-package-test-inputsexit 0

Two gates initially refused on unbuilt workspace dependenciescheck:skill-examples on @objectstack/client-react, and check:doc-formula-expressions on @objectstack/formula — a pre-existing worktree state, not this change. Both refuse loudly rather than reporting a false green ("Nothing was measured"), and both pass after building the closure. check:skill-docs and check:skill-refs are green without regeneration, so the strip moved nothing those generators own.

No changeset — skip-changeset applied.pr-automation.yml's own prescription names skills/ on the "releases nothing" list and explicitly closes the empty-changeset route: "skills/* is on that list, and it is spelled out because the git log says otherwise."* skills/ is not a workspace member and no package's files includes it; the other half of the diff is repo tooling under scripts/.

Out-of-scope findings filed

Out of scope, and left open: #11791 (bare ADR numbers, 177 sites) is a separate sub-issue of the same parent, measuring a different citation kind.


Generated by Claude Code


Generated by Claude Code

…and gate their return
`skills/**` ships to customer projects and is loaded WHOLE into customer agent
context windows. A `#NNNN` there resolves to nothing for that reader -- a
citation-shaped token billed to every customer session, forever.
Maintainer ruling 2026-08-23 (option A), resting on the standing ruling of
2026-08-12, verbatim and untranslated:
「处理 issue 时犯的错应该总结成经验,保留 issue id没有意义」
- Strip 129 id references across 20 hand-authored published files, keeping the
teaching and dropping the citation. Where a customer-resolvable anchor
existed it replaces the id (protocol version, ADR number, lint rule id).
- Add a second corpus rule to `check-doc-authoring.mjs` banning bare internal
ids on the customer-facing surface, with its own walker (the bare-literal
walk skips `references/`, where a ninth of the population lived), a
generated-artifact exemption, and a red/green self-test plus precision cases
for hex colours, version numbers, HTTP codes, array indices and the `#1`
ordinal.
- Lower all ten shrunk token-ratchet ceilings to the re-measured readings;
bundle 117943 -> 117725 tokens (-218).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RMTpSRF5CjMmQBFfPtPCwJ
@claudeclaudeBot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 25, 2026
@github-actionsgithub-actionsBot added size/l documentation Improvements or additions to documentation labels Aug 25, 2026
…, and classify the gate
`check:ratchet-remedy-authority` was RED on the first cut: the new rule's failure
text handed the author a remedy that EXPANDS a registry (an EXAMPLE_ID_ALLOWLIST
entry) while neither marking that path ⛔ MAINTAINER-ONLY nor turning it down.
Maintainer ruling 2026-08-25 took the stronger route -- option B -- which
dissolves the finding at the root rather than marking it:
- `skills/objectstack-pm-dispatch/SKILL.md` line 70 now reads
`/pm-dispatch #<n> #<n>`, the same placeholder spelling the sibling
`filed as #<n>:` site in that file already used. The `#` still teaches the
argument grammar; the numbers stop impersonating a citation. Byte-neutral, so
the ratchet ceiling is unmoved (14239).
- EXAMPLE_ID_ALLOWLIST, its predicate, its call site, its four self-test pins
and every mention in the failure text are deleted. The rule now offers no
expansion path at all, and says so.
- Its two self-test cases are replaced by the pair that matters: the `#<n>`
placeholder -- the remedy the failure text prescribes -- must PASS, and the
concrete ids it replaced must stay RED.
Also records check-doc-authoring.mjs in the control corpus of
check-ratchet-remedy-authority.mjs as `excluded` (UNCLASSIFIED, its second
finding): it carries no ratchet, and its path-scoped declarations are never
offered to the author as something to widen.
Side effect worth having: the hand-authored published corpus now contains zero
concrete internal ids, with no exemption anywhere.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RMTpSRF5CjMmQBFfPtPCwJ
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationImprovements or additions to documentationsize/lskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] The published skill catalog cites 92 internal issue ids customers cannot open — and every one is billed to their context window

2 participants

@hotlong@claude