CI: run the generator and assert its output matches what is committed - #30
Merged
Conversation
generate.js sat broken on main for several commits (#28) and nothing caught it, because no job ran it: the Content Validation workflow runs validate.js and stats:check, neither of which loads the generator. A file that produces every entry in data/entries/ and both webapp data bundles had no coverage at all. Adds a "Generator reproducibility" job that runs generate.js and then asserts `git diff --exit-code` over data/entries, docs/data.js and docs/incidents.js. That catches three things the previous jobs could not: the generator failing to load, a hand-edit to a generated file, and a source change that was never regenerated. Also brings the path filters up to date — they still listed three source directories, so nothing under ast-top10/ triggered validation, and a change to generate.js itself triggered nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Without this the workflow cannot verify its own edits — the new Generator reproducibility job did not run on the PR that added it, because .github/ was not in the path filters. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
generate.jssat broken onmainfor several commits (fixed in #28) and nothing caught it — because no job ran it. The Content Validation workflow runsvalidate.jsandstats:check, neither of which loads the generator. A file that produces every entry indata/entries/and both webapp data bundles had no CI coverage at all.What
A Generator reproducibility job that runs
node scripts/generate.jsand then asserts:That catches three failures the existing jobs cannot:
data/entries/and the webapp disagree with the Markdown.Also
The path filters still listed three source directories. Nothing under
ast-top10/triggered validation, and a change toscripts/generate.jsitself triggered nothing. Both are added.Verification
Run locally against the current tree: generator runs clean,
git diff --exit-codeover those paths is empty — the guard passes onmainas it stands, so it gates future drift rather than flagging existing state.🤖 Generated with Claude Code