test+docs: close the gap in #18's fix, refresh README drift named in #19 - #60
Merged
Merged
Conversation
added 2 commits
August 26, 2026 14:41
#18's fix (PR #42) paired the release-please branch shape with a user.type == 'Bot' check in both gate-attestation.yml and hybrid-gate.yml, but nothing exercised it -- actionlint checks the workflow's shape, not what the expression or the shell decides for a given author, and #19 named exactly this gap ("no fixture caller proving ... adversarial cases such as the branch-prefix bypass in #18"). Extracts the live if-expression and run-block text from both workflows (never a hand-copied duplicate, so a future edit is what gets judged, not a description of it) and drives each against a human/fork PR using the release-please branch prefix. Verified against the actual regression: a scratch copy of the pre-#42 if-expression (branch shape alone) makes the spoofed case evaluate to waived=True, so this fixture would have caught it.
…tree #19 named this repo's README as a defect in itself: a nonexistent gate-attestation `runner` input, four of six action pins on a stale version (actions/checkout v6 vs the v7.0.1 every workflow actually pins, setup-rust-toolchain, cargo-deny-action, actions/stale likewise), a caller-pattern section covering 4 of the 10 consumer-facing reusables, and a fleet-rollout list that named 12 repos while a live code-search shows 18 already converted with zero entries for most of them (koinon, mneme, sphragis, zetesis, typikon, thumos, heurema, gnomon among them). A hand-typed table describing a workflow file is a second copy of that file, free to diverge invisibly the moment either side changes without the other -- which is exactly what happened here. scripts/render_readme_tables.py reads the workflow_call blocks and uses: lines directly and (best-effort, network-dependent) the org via GitHub code search, and prints the three tables for splicing back in. This commit is one such splice; re-run the script and re-splice whenever the tree or the fleet's adoption moves. Also fixes the visibility-requirement section's example, which named theatron and logismos as the private consumers this repo's public visibility unblocks -- both are public. gnomon is private and genuinely consumes gate-attestation + release-please today per the same query.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Finding
Two stale-issue findings against this repo:
gate-attestation.ymlandhybrid-gate.ymlnow requireuser.type == 'Bot'alongside the branch shape. But Authenticate release automation; never waive required gates by branch name #18's own "Required mechanism" also asked for "adversarial workflow tests for a human/fork PR using the release-please prefix" and "verify that a spoofed-prefix fixture fails the emitted required check" — nothing did that. No test intests/touches this at all.mainbranch-protection 404 that is now stale (main isprotected: truewithrequired_pull_request_reviews.required_approving_review_count = 1;required_status_checksstill 404s, unchanged), and named specific README drift: a nonexistentgate-attestationrunnerinput, 4 of 6 pinned actions on stale versions, a caller-pattern section covering 4 of 10 reusables, and a fleet-rollout list missing most of the fleet's actual converts.Evidence
gh pr view 42 --json mergedAt,mergeCommit→ merged 2026-08-17,08a0badc7eeb726d59d8a5c419b816693e0305df.gate-attestation.yml:38andhybrid-gate.yml:456-461both gate the branch-shape waiver onuser.type == 'Bot';hybrid-gate.yml'scheck-trailerstep (202-256) has no branch bypass at all.tests/referencedrelease-please,PR_AUTHOR_TYPE, orspoofbefore this PR (grep -rlnempty).tests/release-please-waiver.shextracts the liveif:expression fromgate-attestation.yml(translated + evaluated, not hand-copied) and the liverun:block fromhybrid-gate.yml'sai-attributionstep, and drives both against a human/fork PR on arelease-please--branches--*branch. All 12 cases pass. Mutation-checked: reverting the extracted expression to the pre-fix(gate): require a Bot author type alongside the release-please branch shape #42 shape (branch shape alone) flips the spoofed case towaived=True, confirming the fixture actually catches the regression it names.gh api repos/forkwright/.github/branches/main/protection/required_pull_request_reviews→required_approving_review_count: 1..../protection/required_status_checks→ 404"Required status checks not enabled".actionlint.yml:34-37still pipescurlstraight intotar, unchanged (line numbers shifted slightly from Make reusable-workflow publication a protected executable release boundary #19's original 26-32 citation as the file grew).uses:lines (13actions/checkoutsites, allv7.0.1@3d3c42e5...; README saidv6@de0fac2e...; same story foractions-rust-lang/setup-rust-toolchain,EmbarkStudios/cargo-deny-action,actions/stale, andSwatinem/rust-cache's SHA — onlygoogleapis/release-please-actionmatched).gate-attestation.yml:6isworkflow_call: {}— zero inputs — while README documented arunnerinput for it."forkwright/.github/.github/workflows" org:forkwright, filtered to.github/workflows/paths) finds 18 repos currently consuming a reusable from this repo; the README's "Remaining fleet rollout" named 12 repos total (both lists combined) and omitted koinon, mneme, sphragis, zetesis, typikon, thumos, heurema, gnomon entirely.gh repo list --json isPrivate). gnomon is private and genuinely consumesgate-attestation+release-pleaseper the same code-search query.Why this matters
A checker never shown failing is not a checker — #18's fix had no regression test, so a future edit that reintroduces the branch-only trust (or narrows the
user.typecheck) has nothing to catch it before it ships. And a README that is the CI-authoring repo's own onboarding doc, read stale, sends every new consumer to copy arunner:input that silently does nothing and to trust action pins two majors behind what's actually running.Desired correction
tests/release-please-waiver.sh: adversarial fixture for Authenticate release automation; never waive required gates by branch name #18, extracting and driving the live workflow logic rather than a hand-copied description of it.README.md: regenerated Workflow inputs / Pinned action versions / Fleet rollout sections,scripts/render_readme_tables.pyto keep regenerating them, and the two caller-pattern/visibility-requirement inaccuracies fixed inline.Done when:
bash tests/release-please-waiver.shexits 0 (verified above) and README's three generated sections matchpython3 scripts/render_readme_tables.py's current output (verified above, this PR's docs commit IS that splice).Not in this PR
mainas a release boundary, checksum-verified actionlint install, executable fixture callers for every reusable's contract, a publication receipt, kanon#2565 pin-wave promotion, CI-enforced drift rejection) stays open — that is a substantial standalone mechanism referenced by forkwright/kanon#2293 and #2565, not a docs fix. This PR narrows Make reusable-workflow publication a protected executable release boundary #19's evidence to what's still true and fixes what it named about the README; Make reusable-workflow publication a protected executable release boundary #19 itself should stay open.mainrequiring status checks) are reported separately, not touched here — this fleet's policy reserves protection mutations for the orchestrator.curl | tarinstall is Make reusable-workflow publication a protected executable release boundary #19's scope, not touched here.Closes#18. Refs #19 (evidence refreshed, kept open -- see comment on #19).