Skip to content

Keep machine constants off the skill-reference Exports: fallback - #12391

Merged
os-zhuang merged 2 commits into
mainfrom
claude/issue-12201-exports-fallback-ranking
Aug 26, 2026
Merged

Keep machine constants off the skill-reference Exports: fallback#12391
os-zhuang merged 2 commits into
mainfrom
claude/issue-12201-exports-fallback-ranking

Conversation

@os-zhuang

@os-zhuangos-zhuang commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Closes#12201

Implements the recorded adjudication (comment 5412452726, re-dispatched at 5418442483): ranking A — skip SCREAMING_SNAKE, keep source order for what remains. ⛔ Schema-first reordering is NOT taken; ⛔ the loud-refusal option (require module doc blocks, drop the fallback) is NOT taken, and no module doc blocks were authored.

The defect

When a .zod.ts has no module doc block, extractDescription() falls back to an export list. The line is true — an accurate list of what the module exports, which is why #12094 kept it. What is wrong is the ranking: rank was source order, and the extraction had no notion of authorable surface, so slice(0, 5) kept whichever five exports were declared first. Three of the eleven modules that reach the fallback declare machine constants near the top and therefore headlined them, on a surface loaded whole into a customer agent's context window to teach it what it may author.

check:skill-refs could not see it: that gate compares the artifact against the generator, and the generator ranked faithfully.

Before / after — all three rows, verbatim

skills/objectstack-automation/references/_index.md

-- `node_modules/@objectstack/spec/src/automation/approval.zod.ts` — Exports: ApproverType, DEPRECATED_APPROVER_TYPES, NON_AUTHORABLE_APPROVER_TYPES, ORG_MEMBERSHIP_LEVELS, APPROVER_EXPRESSION_ROOTS+- `node_modules/@objectstack/spec/src/automation/approval.zod.ts` — Exports: ApproverType, ApprovalDecision, ApprovalNodeApproverSchema, DecisionOutputDefSchema, ApprovalEscalationSchema

skills/objectstack-i18n/references/_index.md

-- `node_modules/@objectstack/spec/src/system/translation.zod.ts` — Exports: LocaleSchema, FieldTranslationSchema, ActionResultDialogTranslationSchema, ObjectTranslationDataSchema, LEGACY_OBJECT_FIRST_KEYS+- `node_modules/@objectstack/spec/src/system/translation.zod.ts` — Exports: LocaleSchema, FieldTranslationSchema, ActionResultDialogTranslationSchema, ObjectTranslationDataSchema, TranslationDataSchema

skills/objectstack-platform/references/_index.md

-- `node_modules/@objectstack/spec/src/kernel/plugin.zod.ts` — Exports: PluginContextSchema, CORE_PLUGIN_TYPES, CONSUMER_INSTALLABLE_TYPES, PluginSchema+- `node_modules/@objectstack/spec/src/kernel/plugin.zod.ts` — Exports: PluginContextSchema, PluginSchema

ApproverType stays first on the headline row — the property the adjudication turned on, and the reason Schema-first was rejected.

The cap of five is applied after filtering, not before. Slicing first would let a module's constants consume the row's five slots and then be deleted from it, shortening the hazardous rows instead of promoting the authorable names behind them (translation would publish four names where five were available).

Acceptance scan, with a positive control shown able to fire

The scan is not a throwaway grep — it is the corpus gate at the end of packages/spec/scripts/export-list.test.ts, so it ships and cannot rot. It enumerates every Exports: row in the checked-in skills/** artifacts and fails on any SCREAMING_SNAKE name.

Positive control, run pre-fix (new test file against the unmodified generator and unmodified artifacts) — it fires, naming all 7 offenders:

FAIL scripts/export-list.test.ts > published catalog — no Exports: row names a machine constant
AssertionError: expected [ …(7) ] to deeply equal []
+ "skills/objectstack-automation/…/_index.md: …/automation/approval.zod.ts → DEPRECATED_APPROVER_TYPES",
+ "skills/objectstack-automation/…/_index.md: …/automation/approval.zod.ts → NON_AUTHORABLE_APPROVER_TYPES",
+ "skills/objectstack-automation/…/_index.md: …/automation/approval.zod.ts → ORG_MEMBERSHIP_LEVELS",
+ "skills/objectstack-automation/…/_index.md: …/automation/approval.zod.ts → APPROVER_EXPRESSION_ROOTS",
+ "skills/objectstack-i18n/…/_index.md: …/system/translation.zod.ts → LEGACY_OBJECT_FIRST_KEYS",
+ "skills/objectstack-platform/…/_index.md: …/kernel/plugin.zod.ts → CORE_PLUGIN_TYPES",
+ "skills/objectstack-platform/…/_index.md: …/kernel/plugin.zod.ts → CONSUMER_INSTALLABLE_TYPES",

(The card named two constants on the approval row; the row actually carried four — four of its five slots were unusable by an author.)

After the fix, the same gate is green, and grep -rn '— Exports:' skills/ returns 14 rows, none naming a machine constant.

Token delta — two readings, because the ratchet cannot see this change

node scripts/check-skills-token-ratchet.mjs✓ check-skills-token-ratchet: 11 published SKILL.md within their ceilings. with bundle total 117851 / 117857 (-6)identical before and after this diff, because that gate's population is skills/<name>/SKILL.md only and this change touches no SKILL.md. Reporting only that number would be reporting a metric blind to the change.

Whole-tree skills/**, same convention (ceil(utf8 bytes / 4)), 49 files both sides:

bytestokens
before745007186252
after744946186237
delta−61−15

Per changed file (lines unchanged at 44 / 35 / 62):

filebytestokens
objectstack-automation/references/_index.md3144 → 3133786 → 784
objectstack-i18n/references/_index.md2045 → 2042512 → 511
objectstack-platform/references/_index.md5048 → 50011262 → 1251

Filtering shrinks the catalog, as expected — no expansion budget is consumed.

Test, and its reverse verification

packages/spec/scripts/export-list.test.ts extends the pattern #12202 used: that PR carried no test of its own — it inherited coverage by importing the shared findModuleDocBlock(), whose pin is file-description.test.ts, which asserts on an extracted pure module because the generator runs main() on import. The same move is made here: the ranking rule moves to packages/spec/scripts/lib/export-list.ts and is pinned without running the generator.

Six unit cases plus a two-case corpus gate; 8 passed at the final commit. The two halves fail differently, which is why both exist:

  • Ablation — dropping the MACHINE_CONSTANT test from exportListDescription, everything else kept: 4 of the 6 unit cases go red (the three about which names survive, plus the fall-through case, whose null exists only because filtering can empty a list), while the corpus gate stays green — it reads checked-in artifacts, which only move when someone regenerates. What reddens the corpus gate is regenerating with the rule dropped, i.e. the pre-fix measurement above.
  • No dist/ is involved and none was needed: the test imports ./lib/export-list by relative specifier inside the same package's scripts/ directory, so vitest resolves the TypeScript source — no exports map, no build step between the edit and the reading.
  • The mutation was confirmed on disk before the reading, anchored on the text being changed rather than on a bare diffstat: guarded-push grep -c 1 → 0, unguarded-push 0 → 1, git diff --stat1 insertion(+), 1 deletion(-). The script carried trap '<restore>' EXIT INT TERM; restoration was confirmed byte-identical (git diff HEAD -- <lib> empty) before the amend.

Three-layer input declaration (found by the gates, not by me)

The corpus gate reads outside its own package, so check:cross-package-test-inputs reddened — correctly. Repaired by declaring the input, never by moving the fixture, across all three layers the mechanism has:

  1. scripts/cross-package-test-inputs.mjs'skills/**' on @objectstack/spec
  2. turbo.json — the mirrored $TURBO_ROOT$/skills/** input on @objectstack/spec#test
  3. .github/workflows/ci.yml — the same glob verbatim in the crosspkg: filter, which check-ci-filter-parity then required (layer 3 is the scheduler, and neither of the other two can see it)

The subtree rather than skills/*/references/_index.md: the test reads the directory too, and a file-only glob does not cover a directory listing (coversDirectory is the check). The narrower spelling was tried first and rejected by the gate.

Gates

Derived with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack, re-derived after the edits — the second derivation added five families (check:bash32-floor, check:cli-command-ids, check:entry-guard, check:parse-guard, check:pnpm-filter-targets) once the diff grew scripts/**, and those were run too. All run at final commit 74667b86d, exit codes captured before any pipe.

  • pnpm --filter @objectstack/spec run check:skill-refs✅ 9 generated files in sync with packages/spec
  • pnpm --filter @objectstack/spec test scripts/export-list.test.tsTest Files 1 passed (1) · Tests 8 passed (8)
  • pnpm check:cross-package-test-inputsOK: 18 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.
  • node scripts/check-ci-filter-parity.mjs → exit 0 (was the layer-3 failure above)
  • node scripts/check-skills-token-ratchet.mjs✓ check-skills-token-ratchet: 11 published SKILL.md within their ceilings.
  • pnpm check:nul-bytes✓ check-nul-bytes --self-test: 75 assertions over a temp git repo (real scan() path)
  • pnpm --filter @objectstack/spec typecheckcheck:test-typecheck: OK — @objectstack/spec's test layer compiles under packages/spec/tsconfig.test.json
  • pnpm check:doc-authoring✓ doc authoring guard: 48 published skill files clean — no internal issue-id references.
  • pnpm check:skill-frame-sync✓ check-skill-frame-sync: 4 copies of the decision frame are structurally isomorphic across 3 files
  • pnpm check:type-check-coverage✓ check:type-check-coverage --self-test — 47 semantic case(s) + 65 observation case(s) + 29 re-measure case(s) + 28 built-closure case(s) + 19 auto-lowering case(s) hold.
  • pnpm check:where-matcher✓ where-matcher conformance holds: 302 matcher(s) discovered, 302 answer the combinator battery correctly or refuse it loudly (189 refuse).
  • pnpm --filter @objectstack/lint run check:doc-formula-expressions → exit 0, after building its @objectstack/formula + @objectstack/lint closure (it refuses outright on an unbuilt tree and says so)
  • Also green: check:agent-test-spelling, check:role-word, check:pm-governed-merges, check:skill-compatibility, check:changeset-gate-self-tests, check:objectui-changeset, check:merge-driver, check:published-files, check:page-declaration-shape, check:slot-lookup, check:test-source-alias, check:type-source-resolution, check:engine-double-contract, check:query-options-erasure, check:bash32-floor, check:cli-command-ids, check:entry-guard, check:parse-guard, check:pnpm-filter-targets, check:empty-state, check:liveness, check:strictness-ledger, check:variant-docs, check-adr-0087-registration, check-changeset-no-major, check-empty-changeset, check-plugin-teardown-shape, release-rehearsal-clone --self-test, docs-audit/check-affected-docs, docs-audit/check-drift-comment.

Declared narrowing — two families not run:

  • node scripts/check-dev-prereqs.mjs exits 1 in this fresh worktree because 67 of 67 workspace packages have no dist/ on disk. Its own remedy is pnpm build — a whole-workspace build CI owns. It measured nothing about this diff; the count being all packages is what shows it is an environment fact, not a consequence of this change.
  • pnpm check:type-check-debt, the convention-triggered ratchet for adding a test file, refuses on an unbuilt workspace closure for the same reason. Its spec-level equivalent, check:test-typecheck, ran green above and is the half that judges this diff's new test file; the repo-wide --re-measure is CI's.

Repo-wide pnpm lint is CI's run, not narrowed here — no claim is made about it either way.

Merge regime — governed, measured not assumed

PR #12274 (the #11705 generated-artifact exemption) is still open, not merged, so the pre-exemption regime applies. Measured rather than asserted:

$ node scripts/pm/check-governed-merges.mjs --test <the 10 files>
⛔ GOVERNED — a human merge is the review record for this PR (#9495 regime).
skills/** ×3 — the published skills catalog
EXIT=3

Worth noting for whoever reviews after #12274 lands: this PR would stay governed even then. Its own table puts a diff that edits the generator alongside its artifacts under the #11084 co-edit fence — "the diff edits the instrument that would certify it" — and that is exactly this diff.

Draft regardless; PM reviews. No seat flips it ready, enqueues it, or arms auto-merge.

Changeset present (@objectstack/spec: patch), matching what #12202 did on its merged diff — this is a user-visible change to published artifacts, so skip-changeset does not apply.

Generated by Claude Code


Generated by Claude Code

…fallback
The `Exports: …` line a pointer row falls back to when its module has no doc
block ranked by SOURCE ORDER and had no notion of authorable surface, so
`slice(0, 5)` kept whichever five exports were declared first. Three of the
eleven modules reaching that fallback declare machine constants near the top,
so three published rows headlined them — `DEPRECATED_APPROVER_TYPES` and
`NON_AUTHORABLE_APPROVER_TYPES` among them, on a surface loaded whole into a
customer agent's context window to teach it what it may author.
SCREAMING_SNAKE exports are now dropped and source order kept for the rest,
with the cap of five applied AFTER filtering so the authorable names behind the
constants are promoted rather than the row merely shortened. A module whose
whole export surface is machine constants falls through to no description
rather than a bare `Exports:`.
Schema-first sorting was considered and NOT taken: on the row that motivated
this it demotes `ApproverType`, the enum an author actually writes.
The rule moves to `scripts/lib/export-list.ts` so it can be pinned without
running the generator (`main()` runs on import), and `export-list.test.ts`
enforces it as unit cases plus a corpus gate over the checked-in artifacts.
Closes#12201
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RMTpSRF5CjMmQBFfPtPCwJ
@github-actions

github-actionsBot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json e91e62acba1fc0d165031a4e561a39dc20634155packageMentionDocs.

…-> 14, by name)
The pin is judged over the live declaration table by design; the skills/**
declaration this PR adds moves the rollback count, so it is recorded by name
per the pin's own convention. Both legs green: self-test 40 assertions, main
check 105/84 globs covered.
@os-zhuang
os-zhuang added this pull request to the merge queueAug 26, 2026
Merged via the queue into main with commit cdbd920Aug 26, 2026
34 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-12201-exports-fallback-ranking branch August 26, 2026 01:01
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cddocumentationImprovements or additions to documentationsize/mteststooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] The skill-reference Exports: fallback ranks by source order, so DEPRECATED_/NON_AUTHORABLE_/LEGACY_ constants headline customer-facing rows

2 participants

@os-zhuang@claude