ci: enforce safer pull request evidence - #716
Merged
Merged
Conversation
Add trusted PR evidence validation and exact default-branch CI failure attribution. Document and self-test the safer review workflow.
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This pull request has been ignored for the connected project Preview Branches by Supabase. |
Contributor
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughAdds a trusted PR metadata policy workflow, shared CI triage helpers, self-tests, PR template guidance, and documentation updates covering verification, risk/rollback evidence, CI attribution, and process governance. ChangesPR policy enforcement
CI triage
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant PullRequest
participant PRPolicyWorkflow
participant BaseCheckout
participant PRPolicyEvaluator
PullRequest->>PRPolicyWorkflow: pull_request_target metadata
PRPolicyWorkflow->>BaseCheckout: checkout base SHA
PRPolicyWorkflow->>PRPolicyEvaluator: evaluate PR metadata and changed files
PRPolicyEvaluator-->>PRPolicyWorkflow: classification and errors
PRPolicyWorkflow-->>PullRequest: policy summary and result
sequenceDiagram
participant CIWorkflow
participant GitHubActionsAPI
participant CITriageHelpers
participant PullRequest
CIWorkflow->>GitHubActionsAPI: retrieve failed jobs
CIWorkflow->>CITriageHelpers: classify failures against default-branch baseline
CITriageHelpers-->>CIWorkflow: generated triage body
CIWorkflow->>PullRequest: post triage comment
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
scripts/ci-triage.mjs (1)
63-107: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueSelf-test coverage gap:
selectLatestDefaultBranchRunreturningundefined.The self-test covers the happy path for all four exports but doesn't test
selectLatestDefaultBranchRunwith an empty or no-match array (returnsundefined), norclassifyFailedJobswith anullmainRun. Both paths are handled correctly in production code, but adding assertions would lock in that behavior.♻️ Optional: add edge-case assertions
assert.equal(selectLatestDefaultBranchRun(runs, { currentRunId: 99, defaultBranch: "main" })?.id, 2); + assert.equal(selectLatestDefaultBranchRun([], { currentRunId: 99, defaultBranch: "main" }), undefined); assert.deepEqual( failedJobNames([ { name: "Build", conclusion: "failure" }, { name: "Lint", conclusion: "success" }, ]), ["Build"], ); assert.deepEqual(classifyFailedJobs(["Build", "Lint"], runs[0], ["Build"]), [ { name: "Build", classification: "main-side" }, { name: "Lint", classification: "needs-investigation" }, ]); + assert.deepEqual(classifyFailedJobs(["Build"], null, []), [+ { name: "Build", classification: "needs-investigation" },+ ]);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/ci-triage.mjs` around lines 63 - 107, Add self-test assertions for the no-match or empty-array case of selectLatestDefaultBranchRun, verifying it returns undefined, and for classifyFailedJobs with a null mainRun, verifying the expected needs-investigation classifications. Keep the existing happy-path assertions unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/pr-policy.mjs`:
- Around line 113-118: Update the clinical-risk validation in the
classification.clinicalRisk branch of scripts/pr-policy.mjs to validate every
required item in the Clinical Governance Preflight checklist defined by
.github/pull_request_template.md, rather than only detecting unchecked boxes.
Require all expected checklist items to be present and checked, and retain the
existing missing-section error behavior.
- Around line 121-130: Update the risk-and-rollout validation in the
classification check around the Risk and Rollback regexes to reject bare
severity values such as “Risk: low” unless they include substantive rationale,
and reject no-op rollback responses such as “Rollback: none..”. Preserve
acceptance of concrete risk explanations and actionable rollback plans while
enforcing the existing error messages.
- Around line 63-74: Update branchLikeTitle so it no longer classifies titles
solely because value contains "/". Retain the codex/claude/copilot prefix
detection and normalized equality check against headRef, allowing
outcome-focused titles such as “fix: handle /api/search failures”.
---
Nitpick comments:
In `@scripts/ci-triage.mjs`:
- Around line 63-107: Add self-test assertions for the no-match or empty-array
case of selectLatestDefaultBranchRun, verifying it returns undefined, and for
classifyFailedJobs with a null mainRun, verifying the expected
needs-investigation classifications. Keep the existing happy-path assertions
unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: e8cf2745-e933-4eab-a91f-682ddaf4c2e6
📒 Files selected for processing (11)
.github/pull_request_template.md.github/workflows/ci-triage.yml.github/workflows/ci.yml.github/workflows/pr-policy.ymldocs/branch-review-ledger.mddocs/codex-review-protocol.mddocs/process-hardening.mdpackage.jsonscripts/ci-change-scope.mjsscripts/ci-triage.mjsscripts/pr-policy.mjs
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Require the full governance checklist, substantive risk and rollback evidence, and preserve valid slash-bearing outcome titles. Cover CI triage fallback paths.
Uh oh!
There was an error while loading. Please reload this page.
This was referenced Jul 17, 2026
Merged
This was referenced Jul 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
main.Verification
npm run check:pr-policy— self-test passednpm run check:ci-triage— self-test passednpm run check:ci-scope— self-test passednpm run check:github-actions— immutable pin check passedgit diff --checknpm run verify:pr-localVerification not run:
npm run verify:pr-localwas repeatedly blocked by the repository's cross-worktree heavyweight lock while other registered Database tasks were building/testing. Hosted exact-head CI is the required broad gate before merge.Risk and rollout
pull_request_targetjob has read-only permissions, checks out the exact trusted base SHA, and never executes PR-head code.4983a7f31; the independent repository permission settings can be reversed through GitHub settings if they cause an unexpected workflow incompatibility.Clinical Governance Preflight
Not applicable: this changes repository process metadata and CI orchestration only; no clinical, retrieval, source, privacy, document-access, or production application behavior changes.
Notes
Summary by CodeRabbit