Skip to content

ci(changeset): report changesets a change modified but did not add - #6435

Merged
os-warren merged 1 commit into
mainfrom
claude/issue-6336-changeset-overwrite-guard
Aug 26, 2026
Merged

ci(changeset): report changesets a change modified but did not add#6435
os-warren merged 1 commit into
mainfrom
claude/issue-6336-changeset-overwrite-guard

Conversation

@os-warren

@os-warrenos-warren commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Fixes#6336

A hand-picked changesets-style filename can land on one that already exists, and the overwrite is silent. This adds a report-only gate that names it.

Why this is worth a gate rather than a guideline

The cost lands on a THIRD PARTY and is invisible at the time it happens. The agent that picks a colliding name loses nothing; whichever earlier PR's release declaration vanishes pays, and only discovers it when a package silently fails to bump. Both signals that should catch it fail:

  1. git status shows M, not ??. An agent checking "did my new file appear" sees a modification and reads it as its own write landing. Nothing says you replaced someone else's file.
  2. The loss is a deleted release declaration, which nothing downstream flags — the packages simply do not get bumped, with no red anywhere.

The base rate is real: 424 accumulated changeset files against an adjective-animal-verb name space.

⭐ What actually caught the near-miss on #6165 was neither a gate nor a test: check-changeset-presence reported "0 changeset(s) added", and the dev read that gate's own detection logic instead of assuming the gate was wrong. That thin thread is what this PR replaces.

⭐ The historical measurement — it chose report-only over blocking

Triage ruled "report-only first if the population has legitimate modify cases, and measure that before choosing." Measured over all 5281 first-parent commits on main (2026-01-13 .. 2026-08-25, full unshallowed history), git log --first-parent --diff-filter=MD -- ':(glob).changeset/*.md':

commitsfilesverdict
MODIFY a pre-existing changeset121919/19 legitimate
DELETE a pre-existing changeset88164482 release consumption, 5 hand deletions, 1 hand-versioned release

Every one of the 19 modifications was legitimate:

  • bump-level corrections (11 files / 4 commits) — c3194207d "the pending release is 17.2.0, not 18.0.0" (4 files major to minor), 8f82c6120 "pending changesets stay minor" (5 files), 9dc88f7d9major to patch after review, e7ab048d7minor to patch;
  • factual corrections to prosee222b24c7 "eleven" to "ten" locale packs, 606105310 a typo'd package name @objectstack/console corrected to @object-ui/console;
  • authors amending their own not-yet-released changeset after the change grew — 8c0d52ece, d22ae31ce, bebaebd39, cbfbeeab8, 8feed91c9, cc4ca459d.

⇒ The guard's premise — a PR modifying a pre-existing changeset is almost always a mistakeis not true of this repository's history, 19 for 19 against. A blocking gate would have failed every one of those PRs. Hence report-only, per the ruling. OS_CHANGESET_OVERWRITE_ENFORCE=1 flips it, so revisiting it is a decision rather than a rewrite.

The 88 deletions are dominated by the release itself: 82 delete changesets alongside a package CHANGELOG.md, which is changeset version emptying the queue. The gate recognizes that shape and reports it as the non-finding it is. The 5 hand deletions are supersessions (4 delete a declaration and add a replacement in the same commit) plus 914c3900e "删除 slow-masks-unite.md", a bare deletion of @object-ui/console: minor.

Control terms — every zero-hit and every count was controlled

controlexpectationmeasured
--diff-filter=A on ':(glob).changeset/*.md'pathspec + --first-parent diff plumbing sees changesets at all2068 added files
--diff-filter=M on package.jsonthe M filter is not silently empty301 commits
--diff-filter=D, string chore: release packagesa known-present subject the same sweep must match47 commits, incl. one deleting 291 files
.changeset/README.md in the MD sweepdocumentation must never appear0 — correctly absent

None of the counts above is a bare zero, and the one structural zero (README) is proven by the same sweep that returns 1663 non-zero file hits.

⭐ A narrower signal the history does not contradict

Of those 19 legitimate modifications, 18 keep every package NAME they declared at base — a bump level changes, prose changes, names do not disappear. The single exception dropped @objectstack/console, a name that resolves to no package in this workspace, because that was the typo being fixed. An overwrite is the opposite shape: the previous declaration's names are simply gone. The gate therefore reports lost declarations separately and loudly. Deliberately not built into a blocking rule today — one explainable hit is not a measurement, and the ruling was report-only first.

What ships

  • scripts/check-changeset-overwrite.mjs — reports every .changeset/*.md the change modified or deleted without having added it. base is the merge base, so "pre-existing" needs no ownership bookkeeping: a changeset added and then edited on the same branch is an A, never an M. It prints the declaration that was there, what the file declares now, and what is gone from it.
  • changeset-guard.yml gains a second job, Changeset Overwrite Report. A second job rather than a step in no-major because it reads a diff and so needs fetch-depth: 0, which that job does not want. A path filter is normally a false-negative surface; it cannot be here, because a change that modifies or deletes a .changeset/*.md touches .changeset/**by definition.
  • ⚠️Gate closure included (finding(ci): changeset-guard.yml is not in its own paths: filter, so a change to that gate is never exercised by the PR that makes it — 1 of 2 exceptions to a 5-of-7 convention #6321's convention): scripts/check-changeset-overwrite.mjs is listed in bothpaths: filters alongside the workflow's own YAML, so the PR that changes the gate is the PR that runs it. Pinned by a test.
  • scripts/dependabot-merge-gate.mjs classifies the new check name into NOT_A_GATE — caught by dependabot-merge-gate.test.ts, which refuses to let a new check default into silence.
  • content/docs/guide/ci-cd-pipeline.md — inventory row, the verbatim paths: quote, and a new section for the second job.
  • No copy-paste: the base-ref resolver, git diff wrapper and frontmatter reader are imported from check-changeset-presence.mjs rather than becoming a third copy. The second copy of that resolver (in check-i18n-en-drift.mjs) inherited a real defect from its first draft and had to be fixed to match under check-i18n-en-drift 的显式 --base / OS_I18N_DRIFT_BASE 解析不出时会静默跌落到别的提交,拿错基准比对并报绿 #3766.

Two-direction proof

RED direction — the exact #6165 shape planted on the real repository: a heredoc onto .changeset/olive-donkeys-smile.md, which still carries @object-ui/plugin-charts: minor on main. Mutation confirmed on disk before measuring (injected marker count 1, overwritten marker count 0, blob e643a315e… becoming b5d3fc008…).

Report-only run, EXIT=0, the message a reader actually gets:

Compared the working tree with dd76dc3e0 (merge-base with origin/main): 0 changeset(s) added, 1 modified, 0 deleted.
⚠️ This change touches 1 changeset(s) it did not add — they already existed at dd76dc3e0:
M .changeset/olive-donkeys-smile.md
declared at base: @object-ui/plugin-charts: minor
declares now: @object-ui/plugin-grid: patch
⚠️ GONE from the declaration: @object-ui/plugin-charts

Under OS_CHANGESET_OVERWRITE_ENFORCE=1 the same input gives and EXIT=1.

Restore proved, not assumedgit checkout HEAD -- pinned to the file's absolute path, under trap … EXIT INT TERM: the restored blob hashes to e643a315e7aea250f5d6ffeef80a96d348061b3d, identical to the HEAD blob, and git diff HEAD for that path is empty.

NEGATIVE direction — a change that only ADDS a changeset stays green, and green because the gate looked:

Compared the working tree with dd76dc3e0 (merge-base with origin/main): 1 changeset(s) added, 0 modified, 0 deleted.
✅ No pre-existing changeset was modified or deleted.

EXIT=0 both plain and under enforcement. That is this PR itself. ⚠️ The first attempt at this leg passed for the wrong reason — the fixture was untracked, so git diff never saw it and the gate reported "0 added". Fixed by counting untracked changesets into the added set (parity with check-changeset-presence, which goes out of its way to do the same) and by asserting the "1 changeset(s) added" line, so a green that comes from looking at nothing fails.

Ablation — blinding the gate to modifications (the M listing replaced by an empty array) turns 5 of 21 tests red; restoring returns 21/21 green. Mutation and restore both proved by git hash-object: cbb92c408… to 759cc6747… and back to cbb92c408…, plus marker counts on disk before either measurement. No build step is involved — the gate is a plain .mjs executed from source by both node and vitest, so there is no dist/ for a stale artifact to hide in.

Verification

Union run at ebc678d57 (the final commit):

  • npx vitest run --project unit scripts/__tests__82 files / 2338 tests passed. Run whole, per the warning that ci-cd-pipeline-doc.test.ts pins that page against .github/workflows/ in both directions; it is also what caught the dependabot-merge-gate coupling above.
  • node scripts/check-control-bytes.mjs✅ OK (scanned 5322 tracked text file(s)) (5319 before the commit; the three new files are covered).
  • node scripts/check-entry-guard.mjs✓ 50 scripts/ file(s) … 45 export bindings, 45 of them inert on import.
  • node scripts/check-changeset-presence.mjs, check-changeset-no-major.mjs, check-changeset-fixed.mjs, check-doc-links.mjs, check-doc-fence-languages.mjs, check-doc-component-types.mjs, check-docs-route-eager-closure.mjs, check-lint-coverage.mjs — all exit 0 on their own verdict lines.
  • npx eslint on the three changed code files — exit 0. npx tsc -p tsconfig.scripts.json --noEmit — exit 0, and --listFiles confirms it actually includes the new test file rather than excluding it.

Out of scope by ruling: direction 2 from the card — documenting the convention in AGENTS.md — is not touched here. AGENTS.md is governed surface and triage ruled the gate. The convention is documented in content/docs/guide/ci-cd-pipeline.md and in the gate's own failure message instead.


Generated by Claude Code

A hand-picked `changesets`-style filename can land on one that already exists,
and the overwrite is silent in both directions that should catch it: `git status`
reports ` M` rather than `??`, so it reads as your own new file landing, and a
deleted release declaration is flagged by nothing downstream. The cost lands on a
third party — whichever earlier pull request's declaration disappears — and
surfaces only when a package fails to bump.
New `scripts/check-changeset-overwrite.mjs`, run by a second job in
`changeset-guard.yml` (its own checkout at `fetch-depth: 0`, since this one reads
a diff). It names every `.changeset/*.md` the change modified or deleted without
having added it, prints the declaration that was there and what is now gone from
it, and recognizes the release emptying the queue as the non-finding it is.
REPORT-ONLY, measured rather than cautious: across all 5281 first-parent commits
on `main`, 12 commits modified a pre-existing changeset (19 files) and all 19
were legitimate — bump levels corrected when the pending release line changed,
prose corrections, authors amending their own unreleased changeset. Blocking
would have failed every one. `OS_CHANGESET_OVERWRITE_ENFORCE=1` flips it for
whoever revisits that with a new measurement.
The resolver, `git diff` wrapper and frontmatter reader are imported from
`check-changeset-presence.mjs` rather than copied a third time; the second copy
of that resolver inherited a real defect and had to be fixed to match.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CZgmFFzqA9cX8tBMhvpFe
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A hand-picked changeset filename can silently overwrite another PR's changeset — 371 accumulated files, and git status shows M, not ??

2 participants

@os-warren@claude