Skip to content

Price the whole hand-authored skills bundle, not just SKILL.md - #12423

Merged
os-steve merged 1 commit into
mainfrom
claude/issue-12392-token-ratchet-bundle
Aug 26, 2026
Merged

Price the whole hand-authored skills bundle, not just SKILL.md#12423
os-steve merged 1 commit into
mainfrom
claude/issue-12392-token-ratchet-bundle

Conversation

@os-steve

Copy link
Copy Markdown
Collaborator

Fixes#12392

The published-skills token ratchet enumerated only skills/<name>/SKILL.md — 11 of the 49 files that ship — so an author could add thousands of tokens of rules/*.md and truthfully report a +0 package delta against the only number the gate produced. The gate's own header read as though it priced the published bundle as a category, and its ## Enumeration, never a hand list section actively invited the reader to believe nothing in skills/ could land unpriced.

Implements the scope settled in this card's grading comment (2026-08-26).

What changed

One file: scripts/check-skills-token-ratchet.mjs.

  • discoverSkillFilesdiscoverBundleFiles, which walks every published skill directory recursively instead of naming one file per directory. A file inside an existing skill is now as impossible to land unpriced as a whole new skill was before.
  • 27 new CEILINGS rows, each initialized at its measured value on c026b0d2d (recorded as CEILING_BASIS.bundleExtension). Every newly-priced file starts with exactly zero headroom, so this is a ratchet installation, not a budget grant.
  • Generator-owned outputs are excluded from ratcheting but still measured and printed.
  • The report grew from one total to three, because [finding] The published-skills TOKEN ratchet prices only SKILL.md — 37% of the shipped bundle (68,390 tokens) carries no ceiling #12392's headline complaint was that this gate could not produce a whole-tree delta at all.

The boundary, and the one definition of "generator-owned"

A ceiling on a generated file reds on a legitimate regeneration — a spec author moves an indexed headline, the generator faithfully rewrites the index, and the gate stops them. That is the wrong actor: they did not write the bytes and cannot pay by deleting them.

Membership is decided by generatedExceptionFor() imported from scripts/pm/check-governed-merges.mjsnot by a second definition here. That register exists by the #11705 ruling (2026-08-25), which is explicit on the shape: "The exemption is enumerated from the generator, never a hand-copied path list … Extend that registry; ⛔ do not author a second mechanism." Two definitions of "generator-owned" that can drift is the failure this avoids.

The gate header now states the boundary in full — three populations (priced / excluded-and-why / structurally outside), the generator named as the reason for the exclusion, and the reuse's known limit stated rather than hidden: this gate runs in the pre-build lint group with no toolchain, so it can take only the register's cheap half (the narrowing match), not its expensive half (recompute + byte-compare). That residual risk is closed by a tripwire: the excluded set is pinned by name in --self-test, so a hand-authored file matching a generated spelling cannot slip out of pricing without reddening the gate's own self-test.

Whole-tree token readings (this change IS the measurement)

Convention ceil(utf8 bytes / 4), measured on c026b0d2d:

populationfilestokensbeforeafter
SKILL.md11117,850ratchetedratcheted
rules/**, evals/**, authored references/**2752,670unpricedratcheted
generator-owned (_index.md ×9, react-blocks ×2)1115,746unpricedmeasured, not ratcheted
bundle total (whole shipped tree)49186,26663% priced92% priced

Ceiling total moves 117,857 → 170,527. skills/README.md (1,926 tok) is outside the population: it is the catalog's front page, not inside any skill, and itself carries a generated block from build-skill-docs.ts.

No skills/** content is touched by this PR. It prices that surface; it does not edit it.

Positive control — the gate FIRING before it passes

Run with the new enumeration and the old 11-row ceiling map, before adding any rows:

✗ ...: skills/objectstack-ai/evals/README.md is a hand-authored published bundle file
carrying no ceiling, measured now at 315 tokens. ...
✗ check-skills-token-ratchet: 27 of 49 published bundle file(s) failed their check.

Exit 1, 27 reds naming exactly the unpriced authored files, while all 11 generator-owned files came back green and measured. After adding the rows: exit 0.

Reverse verification — three mutations, each proven on disk and restored

The self-test grew 40 → 64 cases. Each leg confirmed the mutation landed (blob hash differs from HEAD) and the restore was clean (git checkout HEAD -- <path>, blob back to HEAD, git diff HEAD empty):

mutationpredictedobserved
walk stops recursingpopulation collapses to the old shape8 of 64 fail — the fixture walk finds 1 not 4; 27 ceilings name files the enumeration cannot find
boundary answers null for everythinggenerated files become unpriced reds11 of 64 fail — 11 _index.md/react-blocks paths reported as carrying no ceiling
boundary answers generated for everythingvacuity: prices nothing, stays green11 of 64 failthe boundary does not swallow the population (got false, want true)

The third is the vacuity direction the card required: a boundary predicate that silently prices nothing fails loudly rather than reporting a clean bundle while measuring none of it.

Gates

Derived from the actual diff via node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (no hand-supplied paths; it read the change set itself). All run at final commit a51236a4, exit codes captured before any pipe:

check:agent-test-spelling · check:bash32-floor · check:cli-command-ids · check:cross-package-test-inputs · check:entry-guard · check:parse-guard · check:pnpm-filter-targets · check-ci-filter-parity · check-cross-package-test-inputs · the gate itself · the gate's --self-testall green.

Plus four the derivation classes as path-unreachable but that I could see were implicated, run anyway: check:ratchet-remedy-authority (still reads this gate's remedy as marked, 155 scripts swept) · bare-root-worklist --self-test · check:nul-bytes · check-governed-merges --self-test (the register this gate now imports) — all green.

No changeset: scripts/-only, matching the convention in recent scripts/-only merges (851b2d8e5, 0b10068c3, c8bf50f5d, 81f8b7ed4, 9646cdbd1 — none carry one). skip-changeset applied.

Out-of-scope finding filed

#12422 — landing this PR moves the token-ratchet row's population from 11 files to 49, which makes two other files' descriptions of it stale: the SKILLS_DIR skills TRIAGE row's why in scripts/pm/bare-root-worklist.mjs, and #12369's measured table (which lists this gate's precise live spelling as skills/*/SKILL.md, 11/11). Not fixed here: the TRIAGE map is shrink-only and #12369 is an open needs-user-decision card holding the adjudication for that exact row among 16 others, so rewriting one of them mid-decision would be adjudicating a slice of the card the maintainer is holding.


Generated by Claude Code

…LL.md
The published-skills token ratchet enumerated only `skills/<name>/SKILL.md`
— 11 files of the 49 that ship. The other 37% (`rules/**`, `evals/**`, the
hand-authored `references/*.md`) carried no ceiling, so an author could add
thousands of tokens of `rules/*.md` and truthfully report a `+0` package
delta against the only number the gate produced.
The population now walks every published skill directory recursively.
Generator-owned outputs (`references/_index.md`, the react-blocks pair) are
excluded from RATCHETING but still measured and printed, because a ceiling
on a generated file reds on regeneration — the wrong actor. Membership is
decided by `generatedExceptionFor()` from `scripts/pm/check-governed-merges.mjs`,
the register the #11705 ruling created for exactly this question, rather than
by a second definition of "generator-owned" that could drift from it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JANH3y7qe3MD8aLaLXci8N
@claudeclaudeBot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 26, 2026
@os-steve
os-steve marked this pull request as ready for review August 26, 2026 02:26
@os-steve
os-steve added this pull request to the merge queueAug 26, 2026
Merged via the queue into main with commit 69d0e18Aug 26, 2026
34 checks passed
@os-steve
os-steve deleted the claude/issue-12392-token-ratchet-bundle branch August 26, 2026 02:44
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/mskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] The published-skills TOKEN ratchet prices only SKILL.md — 37% of the shipped bundle (68,390 tokens) carries no ceiling

2 participants

@os-steve@claude