Uh oh!
There was an error while loading. Please reload this page.
release-train: develop -> staging - #218
Merged
Merged
Conversation
…d (backend#1681) (#212) * feat(inventory): audit rulesets — the protection layer nothing checked (backend#1681) GitHub runs two protection systems. `read_protection()` reads the second one, but only to satisfy the seven classic-expressible properties -- so the guard can tell that a branch is protected, never that a ruleset EXISTS, what it permits, or who may bypass it. Grepping the org's 26 workflows for a ruleset assertion returns nothing. Two things rest entirely on that unaudited layer, and both were drifting: * `allowed_merge_methods: ["merge"]` on every promotion branch -- not expressible in classic protection AT ALL, and the only thing stopping a promotion PR being squash-merged, which collapses the merge-commit ancestry promote-repo.sh's squash guard reads and jams the next prepare. start-training carried NO ruleset whatsoever. Its classic protection was fully compliant, so the existing guard reported it conformant. * the `v*` tag trust root. `release-python` -- four engineers -- held `always` bypass on backend, data-ingestors and tracebloc-py-package, two of which publish ON the tag (PyPI, GHCR). Nothing could see it; it was found by hand. This adds a `ruleset_policy` of two kinds and a per-repo `rulesets` cell: * READS `/rulesets/{id}` per ruleset, not the cheaper endpoints. Load-bearing: `/rules/branches/{b}` -- the one read_protection() already calls -- omits `bypass_actors` ENTIRELY, so asserting an allowlist from it would assert nothing and report a pass. * CLASSIFIES on target + rules, NEVER on name. client's tag ruleset is named "R8 trust root - protect v* release tags" while its five peers use "Protect v* release tags (supply-chain trust root)"; a name-keyed check reports client as missing its trust root. * BYPASS IS AN EXACT ALLOWLIST, not a floor -- an unexpected actor is the finding, which is what turns a future release-python-style grant into a build failure instead of something an audit has to stumble on. * also asserts enforcement==active (an `evaluate` ruleset looks protective and enforces nothing) and the required rule types. No `divergent` state here, deliberately: a ruleset that exists but permits a squash merge, or carries an extra bypass actor, is drift -- not a documented variant -- because that is precisely what the property exists to catch. Verified, not asserted: * selftest 97 -> 111 pass, 0 fail. The new cases assert that a MISSING ruleset, a squash-PERMITTING one, an `evaluate`-only one, an EXTRA bypass actor and a STALE exemption are each findings, that a differently-named tag ruleset still matches, and that an unreadable read is UNREADABLE rather than a silent pass. * One of those tests caught a fail-open I had just written: an early return skipping the read when every cell was `exempt`. It saves an API call and makes "this repo has no rulesets" unfalsifiable -- and a fully-exempt repo is exactly where an unnoticed ruleset would sit forever. Removed, with the reasoning recorded so it is not re-added as an optimisation. * LIVE against all 20 repos: "No drift. Every repo read, every entry matched." * MUTATION: the same run with the tag bypass allowlist pointed at a team nobody holds produces 6 findings -- one per tag-ruleset repo, each naming the unexpected and the missing actor. Green means something because red is reachable. The report line now names the ruleset kinds too; it previously understated its own coverage. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(caller-drift): ruff E702, and stop calling a ruleset outage a protection one (.github#212) Two review findings on this PR: * `quality / ruff` failed E702 (two statements on one line) in a selftest I added. Fixed; `ruff --isolated --select E4,E7,E9,F scripts/` now passes, which is the org gate's exact selection. * Bugbot (Medium): `evaluate_rulesets` appends its read failures into `protection_unreadable`, but the summary and the `::error::` line both said "branch-protection read(s) FAILED ... protection state UNKNOWN". A rulesets-only outage therefore named the wrong control plane and claimed protection was unknown when it had been read successfully. That is exactly the honesty this file exists to enforce, so: wording is now "protection/ruleset", and the bucket carries a comment saying what it holds and why the two share it (both mean "the caller/copy audit for this repo SUCCEEDED, a different control plane did not"). Selftest re-run: 111 pass, 0 fail. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(caller-drift): separate ruleset read-failures from protection; includes_parents=false Bugbot (.github#212): - read_rulesets listed rulesets WITHOUT includes_parents=false, so org/enterprise rulesets folded in and their ids 404'd on the per-ruleset get-by-id, failing the whole audit. Scope the listing to this repo's own rulesets. - ruleset read-failures were funnelled into protection_unreadable, so a rulesets-only outage reported 'protection state UNKNOWN' (and vice versa). Give rulesets their own unreadable bucket and name each layer separately in the summary + ::error::. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(caller-drift): stubs match /rulesets?includes_parents=false (Bugbot #212) The listing stubs matched endswith('/rulesets'); read_rulesets now appends the query string, so strip it before the suffix check. Restores the conforming / missing / weakened ruleset selftest cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
LukasWodka
commented
Aug 11, 2026
ContributorAuthor
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 7716f00. Configure here.
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated promotion by the release train (RFC-0008 D14). Head is the train-managed
release-train/to-stagingbranch (a mirror ofdevelop), so it never collides with a human PR. Merged only when the fr-gate is green.Note
Medium Risk
Touches release-train promotion merge enforcement and supply-chain tag protection auditing; incorrect inventory or API read logic could miss real drift or block merges on false positives.
Overview
Adds a third conformance family alongside callers/copies and branch protection: GitHub rulesets that classic protection cannot express—merge-commit-only on promotion branches and the
v*tag trust root on tag-publishing repos.repo-inventory.ymlgains fleet-wideruleset_policy(match by target + rules, not ruleset name) and per-reporulesetscells (requiredor writtenexempt) for all repos.scripts/caller-drift.pyloads and validates that contract, fetches each repo ruleset via listing + per-id detail (for bypass actors and merge methods), classifies rulesets by shape, and flags missing/weakened rulesets, stale exemptions, wrong merge methods, and exact bypass-allowlist drift. Ruleset API failures use a separate unreadable bucket so they do not abort caller findings or masquerade as branch-protection failures.scripts/tests/caller-drift-selftest.pyadds offline coverage for the main ruleset failure modes and schema gates.Reviewed by Cursor Bugbot for commit 7716f00. Bugbot is set up for automated code reviews on this repo. Configure here.