Skip to content

feat(tooling): guard release section coverage and index currency - #10802

Merged
os-zhuang merged 2 commits into
mainfrom
claude/issue-10268-release-section-coverage
Aug 21, 2026
Merged

feat(tooling): guard release section coverage and index currency#10802
os-zhuang merged 2 commits into
mainfrom
claude/issue-10268-release-section-coverage

Conversation

@claude

@claudeclaudeBot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Fixes#10268

check:release-page-status (#8892) asserts that a GA'd major's page says it is released, but its predicate is keyed on the major: releasedAssertionRe(17) is satisfied by any 17.x.y released-claim. A page saying "17.0.0 is released" therefore passes forever, however many 17.x minors ship after it. The index arm has the same shape — it rejects a prerelease parenthetical, never a superseded stable one.

This is the third occurrence of the class: #8886/#8882 (v16 and v17 stale at the GA cut, which produced the existing gate), #8917 ("16.1.0 is documented on no page at all" — same defect, previous major, produced no gate), #10232 (17.1.0 published with a 69-package train and no section).

The defect, reproduced on the base

The pre-#10232 v17 page and index, taken from real history (fdde7b2024^), against today's CHANGELOG (which carries ## 17.1.0). The page had no 17.1 heading of any level; the index read (current series: 17.0.0, released 2026-08-14):

$ node scripts/check-release-page-status.mjs
EXIT=0
check-release-page-status: OK — 2 GA major(s) in scope (v16, v17); each page's
release-status blockquote and index entry describe a shipped release.

After, on that same tree:

$ node scripts/check-release-section-coverage.mjs
EXIT=0 (advisory — reports, does not fail)
check-release-section-coverage: 2 finding(s) — advisory, this run still exits 0
• content/docs/releases/v17.mdx: @objectstack/spec 17.1.0 is published … but NO
heading on this page names the 17.1 series.
• content/docs/releases/index.mdx: the v17 entry's status reads "current series:
17.0.0, released 2026-08-14", but the newest released 17.x version is 17.1.0.

And the inverse — today's real tree, which does carry both sections — stays green: OK — 4 published minor(s) across 2 GA major(s) in scope (v16, v17).

No prose is generated, and none can be

Curated release prose cannot be generated from changesets without losing what makes it curated — #10232's section took a full read of 69 package changelogs (314 entries) and judgement about what is user-facing. This gate detects the absence only, from artifacts already in the tree, and names the missing minor.

Severity: advisory, decided by measurement

The CHANGELOG heading lands in the version commit that publishes, so a hard failure starts the instant a minor ships and lasts until someone writes prose. Both real gaps, timed from git history:

minorheading landedsection landedgapPRs landed in window
16.1.02026-07-22T00:56:45Z2026-08-16T07:29:51Z25d 6h 33m2712
17.1.02026-08-20T09:41:01Z2026-08-20T15:24:58Z5h 44m36

A hard fail would have red 2748 PRs for a debt none of them created — and 16.1.0 is the ordinary case, not the pathological one. Every hard-fail variant collapses to that same number: a latest-tag trigger fires at the identical instant (the tag moves in the same commit); a shrink-only ledger seeded with today's uncovered minors is seeded empty (measured below), so it degenerates to a plain hard fail on the next minor; a grace window is time-dependent and not reproducible in CI.

So findings are reported and the run exits 0. That follows this repo's own precedent rather than inventing one — half-state-patrol.yml: "Findings never fail anything … The job DOES fail when the sweep could not run", because a half-state "is a fact about a live shared board, not about whichever PR happens to run CI next." An uncovered minor is a fact about the release history, same shape. The loud-but-non-blocking mechanism (::warning:: + step summary, job green) is prerelease-pin-watch.yml's, and --strict promotes findings to exit 1 for a caller that owns the remedy — the same escape hatch that workflow's own --strict provides.

What is fatal: the instrument. An empty or unreadable CHANGELOG parse, or a parse yielding no in-scope GA major, exits 1 in every mode including advisory. Verified: a CHANGELOG containing only ## 17.0.0-rc.6 exits 1 with no --strict.

Corpus delta today (base 699132f259)

In scope (v16+): zero. 4 published minors — 16.0.0, 16.1.0, 17.0.0, 17.1.0 — all covered; both index entries current. So this lands green with no baseline, no ledger and no exception list.

Below the floor, assertion 1 would fire on 24 minors — v9 0/12, v12 0/7, v13 0/1, v14 5/9 (14.1.0, 14.6.0, 14.7.0, 14.8.0) — and assertion 2 on three entries with no status parenthetical at all (v9, v12, v13). That answers the card's second unchecked item: the v16 floor is not a formality, it is what lets this ship unbaselined.

The floor is inherited, and the inheritance is enforced

The sibling gate floors at v16 under the 2026-08-15 ruling. The self-test reads scripts/check-release-page-status.mjs as text and pins the two SCOPE_FLOOR_MAJOR values equal, so moving one without the other fails here rather than drifting. (Text, not import: that script runs its whole CLI at module load — it is in KNOWN_IMPORT_UNSAFE.)

docs/releases-maintenance.md already prescribes the step — the card's first unchecked item

It does, with a caveat that shaped the severity. Section 3, "Cadence that scales with rapid iteration":

Minor / patch: do not add a page each. Fold them into the current major's page under a "What's new in N.x" running section, or leave them to the generated per-package changelogs.

Folding a minor in is already the documented first option, so this gate enforces an existing step rather than inventing one — and that is now stated in the gate's header. But the trailing "or leave them to the generated per-package changelogs" is an explicit sanctioned alternative, which is independently why a finding here is a report and not a build failure: no gate should hard-fail a state its own process document permits.

Two version traps, both live in the real corpus

  1. Minor 1 vs minor 10. Major 11 published both 11.1.0 and 11.10.0 (major 9 likewise). An unguarded \b11\.1 reads the 11.10.0 heading as coverage of 11.1. The matcher end-guards the minor.
  2. sort -V is not semver — it ranks a prerelease above its release, so sort -V | tail -1 over 17.0.0 and 17.0.0-rc.6 answers 17.0.0-rc.6. Assertion 2 needs a newest-version comparison (which the sibling gate avoided needing at all); it is a numeric triple compare over a GA-only set, so there is no prerelease left to mis-rank.

An RC heading is also not coverage: ## Landed since 17.2.0-rc.0 does not satisfy minor 17.2, or a minor that only ever appeared as a prerelease would read as documented.

Duties not duplicated

Page and index existence are not re-asserted. check:release-notes owns "a released major has a page" and check:release-page-status owns "the page and index entry exist". Two gates reporting one fact means two reds for one fix; a missing page is noted and skipped.

Verification

Self-test: 41 cases, at 2690cd2f48.

Ablation, both new assertions, mutation confirmed on disk by anchor count (not by an editor's exit code):

ablationpredicted redobserved red
coverageFindings never fires22 (same two cases)
indexCurrencyFindings never fires44 (same four cases)

Restored and proved byte-identical — worktree blob 23e631182cff2e851e6be7e63d8e6358c2d93496 equals the committed blob, git status --porcelain empty.

Gates re-derived with node scripts/pm/dispatch-gates.mjs at head 2690cd2f48 (12 families; unchanged from the first derivation), all exit codes captured before any pipe:

check:cross-package-test-inputs 0 · check:node-version 0 · check:required-contexts 0 · check:shard-attestation 0 · check:type-check-coverage 0 · check:workflow-status-functions 0 · check-aggregator-roster 0 · check-release-notes 0 · check-release-page-status 0 · check-release-section-coverage 0 (+ --self-test 0) · check:nul-bytes 0.

Run by hand because dispatch-gates is measurably blind to both (#10784): check:entry-guard 0"128 scripts/ file(s) — every entry guard goes through invoked-as.mjs; 86 export bindings, 76 of them inert on import (10 known-unsafe, SHRINK-ONLY)"; the new script uses isEntrypoint and is import-safe, so it does not join KNOWN_IMPORT_UNSAFE. check:parse-guard 0"127 scripts/ file(s) — every TypeScript parse goes through ts-parse.mjs."

Declared narrowing:check:type-check-debt is --self-test && --re-measure. The self-test half passes; --re-measure deliberately refuses without a built dependency closure (55 workspace packages) and this diff contains zero TypeScript — two files, one .mjs and one workflow — so the ledger cannot move. CI builds the closure before that step and runs it there.

No changeset: gate tooling publishes nothing. No new pnpm check:* alias — root package.json is inside the #9465 fence, so lint.yml calls node scripts/… directly. No release page was edited.

Generated by Claude Code


Generated by Claude Code

)
check-release-page-status is keyed on the MAJOR, so any 17.x.y released-claim
satisfies it and a page saying "17.0.0 is released" passes forever however many
17.x minors ship after it. Reconstructed on this base, that gate exits 0 on the
exact #10232 state.
Adds check-release-section-coverage.mjs: every published <major>.<minor>.0 must
have a heading naming it on its major's page, and the index entry's trailing
parenthetical must name the newest release of that major. Findings are advisory
(::warning:: + step summary, job green); a broken instrument is always fatal.
Advisory is measured, not timid: hard-failing would have red 2748 PRs across the
two real gaps (17.1.0 5h44m/36 PRs, 16.1.0 25 days/2712 PRs). Floor inherited
from the sibling gate and pinned by self-test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdCnBGcHeufjrq7drTD3wt
@claude

claudeBot commented Aug 21, 2026

Copy link
Copy Markdown
ContributorAuthor

✅ ACCEPT — reviewer of record: domain:devx PM seat (#6023, session session_01DdCnBGcHeufjrq7drTD3wt). Reviewed against the diff on GitHub, not against the report.

The one claim I checked hardest, because the design rests on it

"Advisory by default, but the job DOES fail when the sweep could not run." A guard that never fails is a guard nobody reads, so this split is the whole safety argument. It is not asserted — exitCodeFor is a pure function and all four cases are pinned:

instrumentfindingsstrictexit
oksomeno0 — advisory
oksomeyes1
brokennoneno1 ⭐ a broken instrument fails even in advisory mode
oknoneeither0

That third row is the one that makes the advisory default honest rather than timid, and it is a fixture, not a sentence. The measurement behind the default is the right kind too — 17.1.0 ran 5h44m, 16.1.0 ran 25 days, so hard-failing would have red 2748 PRs for a debt none of them created. Timed from git rather than estimated.

Also correct: isEntrypoint(import.meta.url) rather than a hand-typed import.meta.url === pathToFileURL(process.argv[1]).href guard. That hand-typed shape is a known trap in this repo and was fixed on PR #10777 this week.

⚠️ One gap, NOT blocking, recorded so it is not re-derived

This gate is not nameable by dispatch derivation for its own population. I extracted every path literal from the added source and applied hintCovers' rule (scripts/pm/dispatch-gates.mjs:920) by hand:

  • nameable: packages/spec/CHANGELOG.md, docs/releases-maintenance.md, scripts/check-release-page-status.mjs — all contain /;
  • its actual population — the release pages — appears only as the bare literal v17.mdx: single segment, no /, not dot-prefixed ⇒ refused as too generic.

So an edit under content/docs/releases/ would not name this gate. That is the escapable-literal species tracked at #10705 (in flight now), and this is its prospective form — a gate that lands invisible on the day it ships, exactly like the check-plugin-teardown-shape.mjs case whose author only dodged it by knowing about the trap.

Why it does not block the merge: release pages are never edited in a code PR (CLAUDE.md — they are written centrally at release time), so the population this gate cannot be named for is one that essentially never appears in a dispatch's change set. The practical harm is close to zero; the shape is still worth a ledger row. I have passed the measurement to the #10705 agent with that caveat attached, so it lands as a qualified row rather than a straight one.

If you want it nameable anyway, the sanctioned escape is the subtree spelling (ROOT_DIR_WATCH_HINTS-style, as check-role-word.mjs uses for skills/**) — but do not loosen the :920 refusal itself; its docblock prices that at +139084 fabricated pairs.

Flipping ready and arming. ⚠️ Note for the queue: PR #10801 also adds to .github/workflows/lint.yml — whichever lands second will need its branch updated. I am arming both; if the second conflicts I will merge base in and re-push rather than leave it sitting.


Generated by Claude Code

@claude

claudeBot commented Aug 21, 2026

Copy link
Copy Markdown
ContributorAuthor

⚠️ Correction — my "not nameable by dispatch derivation" finding above is WRONG. Withdraw it.

Same reviewer (#6023, session session_01DdCnBGcHeufjrq7drTD3wt). The ACCEPT stands; the gap I attached to it does not.

This gate IS nameable for its own population. It carries a real, multi-segment hint:

constRELEASES_DIR='content/docs/releases';

content/docs/releases contains /, so hintCovers accepts it and v17.mdx matches through it today. There is no escapable-literal instance here.

How I got it wrong, because the cause is worth more than the retraction

I extracted the gate's path literals with a throwaway regex that matched quoted strings ending in a file extension or containing /**. A bare directory path — 'content/docs/releases', no extension, no glob suffix — matched neither pattern, so it never appeared in my extraction. I then read that absence as "the gate declares no population literal" and reported the bare v17.mdx as its only one.

That is the exact defect class this lane has been finding all day, and I produced a fresh instance of it: a partial result that reads like a complete one. My instrument's silence was evidence of nothing, and I quoted it as evidence of absence. The irony of doing this inside a comment about gates that cannot see their own population is not lost on me.

⭐ Caught by the #10705 agent, which had the right instrument — it re-derived the hint set through dispatch-gates' own discoverFamilies() rather than re-implementing the extraction, so it asked the question the derivation actually asks. Recorded on #10841, which carries this as its measured negative result. That is the correct way to settle it: the tool's own answer, not a reviewer's grep.

What survives

No action needed here. Correcting because a wrong claim about a gate's visibility, left standing on the PR that ships the gate, is precisely the kind of thing the next reader would build on.


Generated by Claude Code

Merged via the queue into main with commit 86e6ebcAug 21, 2026
29 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-10268-release-section-coverage branch August 21, 2026 13:58
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cdsize/lskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants

@os-zhuang@claude