Skip to content

ci(deep-review): restore fork PR reviews and tighten reviewer confinement - #2955

Merged
teeohhem merged 1 commit into
mainfrom
tom/trim-comments
Aug 21, 2026
Merged

ci(deep-review): restore fork PR reviews and tighten reviewer confinement#2955
teeohhem merged 1 commit into
mainfrom
tom/trim-comments

Conversation

@teeohhem

Copy link
Copy Markdown
Contributor

Summary

Fork PRs have had no deep review since 18 Aug.actions/checkout@v6 added a guard that refuses to check out fork PR code from a pull_request_target workflow, so the job fails at Checkout PR head on every external contribution. We track the floating @v6 tag, so this arrived with no change on our side. Same-repo PRs were unaffected, and the job fails loudly rather than skipping quietly.

This opts in with allow-unsafe-pr-checkout, which is defensible here only because the job never executes the fork's code: there is no install, no build and no test run, so the reviewers only read the tree. The rest of the change tightens what the reviewer can reach, so the opt-in is a considered trade rather than a muted warning.

What changed

ChangeWhy
allow-unsafe-pr-checkout: trueRestores fork PR reviews.
persist-credentials: false on both checkoutsThe default leaves the job token in .git/config. The action's env scrub covers subprocess environment, not files on disk. Nothing in this job pushes.
--strict-mcp-config.mcp.json, .claude/agents and .claude/skills are all tracked, so a PR can ship config for the reviewer itself. --setting-sources user already covers project skills and agents (verified against CLI 2.1.215 with a canary config); .mcp.json is not covered by it.
Narrower tool allowlistDrops gh api, which takes a write method and cannot be constrained by a prefix pattern, and enumerates git by subcommand instead of git:*.
New Materialize prior review comments stepFetches review threads in trusted shell to .deep-review/prior-comments.md, so dropping gh api does not silently cost us the previous-comments reviewer.

Reviewer config is neutralized with flags rather than by removing files. The review skill computes its diff with git diff $BASE against the working tree, so editing the tree first would forge deletions into the diff reviewers see, and would hide a real change if a PR legitimately touches one of those paths.

Not fixed here

  • The reviewer plugin is still pinned by a movable tag (compound-engineering-v3.6.1) while the CLI beside it is pinned by SHA-512. actions/checkout@v6 is a floating tag too, which is what produced this outage.
  • Runs report permission_denials_count: 19, and the action hides tool output, so I could not confirm which calls were already being denied. This change narrows the allowlist further. Worth comparing the Reviewers (N): footer against a pre-change run on a comparable diff before relying on it for a fork PR.

How to test on Vercel preview

N/A — non-UI change.

References

actions/checkout@v6 now refuses to check out fork PR code from a
pull_request_target workflow, so every fork PR has failed at the checkout
step since 18 Aug and external contributors get no deep review. Opt in
with allow-unsafe-pr-checkout, which is defensible only because nothing
in this job executes the fork's code: no install, no build, no tests.
Close the gaps that guard is warning about:
- persist-credentials: false on both checkouts. The default writes a
pull-requests: write token into .git/config, and the action's env
scrub covers subprocess environment, not files on disk.
- --strict-mcp-config. .mcp.json, .claude/agents and .claude/skills are
all tracked, so a PR can ship config for the reviewer itself.
--setting-sources user already drops project skills and agents
(verified against CLI 2.1.215 with a canary config); .mcp.json is not
covered by it.
- Drop Bash(gh api:*), which accepts --method POST and cannot be
narrowed to GET by a prefix pattern. Prior review threads are fetched
in trusted shell to .deep-review/prior-comments.md instead, so
ce-previous-comments-reviewer does not silently leave the roster.
- Enumerate git subcommands rather than Bash(git:*), which permits
git config diff.external plus git diff as arbitrary execution.
Neutralize by configuration rather than by removing the files: the skill
computes the reviewed diff with git diff $BASE against the working tree,
so touching the tree would forge deletions into the diff the reviewers
see, and hide real changes to those paths.
@changeset-bot

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 2e7670d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel

vercelBot commented Aug 20, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
ProjectDeploymentActionsUpdated (UTC)
hyperdx-ossIgnoredIgnoredAug 20, 2026 9:54pm
hyperdx-storybookIgnoredIgnoredAug 20, 2026 9:54pm

Request Review

@github-actionsgithub-actionsBot added the review/tier-1 Trivial — auto-merge candidate once CI passes label Aug 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🟢 Tier 1 — Trivial

Docs, images, lock files, a dependency bump, or an automated release. No functional code changes detected.

Why this tier:

  • All files are docs / images / lock files

Review process: Auto-merge once CI passes. No human review required.
SLA: Resolves automatically.

Stats
  • Production files changed: 0
  • Production lines changed: 0
  • Branch: tom/trim-comments
  • Author: teeohhem

To override this classification, remove the review/tier-1 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@greptile-apps

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR restores fork deep reviews by opting into fork-head checkout while removing persisted checkout credentials and tightening reviewer configuration and command access.

  • Materializes prior review threads in trusted workflow code after removing reviewer access to gh api.
  • Disables persisted Git credentials for both checkouts.
  • Adds strict MCP configuration and narrows permitted Git/GitHub CLI commands.
  • The new prior-comment handoff is not readable under the resulting tool allowlist, and the new privileged action remains referenced by a mutable tag.

Confidence Score: 4/5

The prior-comment handoff should be fixed before merging because the confined reviewer cannot read the file that replaces its removed GitHub API access.

The workflow writes prior feedback to an ignored untracked file while granting only enumerated Git and GitHub CLI commands, causing the previous-comments reviewer to lose the context this change is intended to preserve; the new mutable action reference also leaves a non-blocking supply-chain concern.

Files Needing Attention: .github/workflows/deep-review.yml

Security Review

The new actions/github-script@v9 invocation adds non-blocking supply-chain exposure because a mutable tag executes with pull-request and issue write permissions.

Important Files Changed

FilenameOverview
.github/workflows/deep-review.ymlRestores fork checkout and tightens reviewer confinement, but the prior-comments file is inaccessible to the confined reviewer and the new privileged action uses a mutable tag.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
Event[pull_request_target] --> Checkout[Checkout fork PR head]
Checkout --> Gate{Review required?}
Gate -->|Yes| Fetch[Materialize prior comments]
Fetch --> File[.deep-review/prior-comments.md]
File -. blocked by tool allowlist .-> Reviewer[Previous-comments reviewer]
Reviewer --> Review[Structured deep review]
Loading

Fix all with GreploopFix All in Claude CodeFix All in ConductorFix All in CursorFix All in Codex

Reviews (1): Last reviewed commit: "ci(deep-review): restore fork PR reviews..." | Re-trigger Greptile

--setting-sources user
--allowedTools "Bash(git:*),Bash(gh pr view:*),Bash(gh pr diff:*),Bash(gh pr list:*),Bash(gh issue view:*),Bash(gh issue list:*),Bash(gh search:*),Bash(gh api:*)"
--strict-mcp-config
--allowedTools "Bash(git diff:*),Bash(git log:*),Bash(git blame:*),Bash(git show:*),Bash(git merge-base:*),Bash(git rev-parse:*),Bash(git ls-files:*),Bash(git cat-file:*),Bash(git status:*),Bash(gh pr view:*),Bash(gh pr diff:*),Bash(gh pr list:*),Bash(gh issue view:*),Bash(gh issue list:*),Bash(gh search:*)"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1Prior-comments file is unreadable

When a PR has prior feedback, the workflow writes it to an ignored untracked file but grants the reviewer only the enumerated Git and GitHub CLI commands. Neither Git nor an allowed file-reading command can read that file, causing the previous-comments reviewer to omit the materialized feedback.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Claude CodeFix in ConductorFix in CursorFix in Codex

- name: Materialize prior review comments
id: prior
if: steps.gate.outputs.should_review == 'true'
uses: actions/github-script@v9

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2securityPrivileged action uses mutable tag

The new actions/github-script@v9 invocation runs in a pull_request_target job with pull-request and issue write permissions, so moving or compromising the mutable tag would allow unauthorized mutation of reviews, comments, or issues. Pinning this invocation to a full commit SHA would make changes to the privileged action auditable. How this was verified: The new action receives GITHUB_TOKEN in a job that explicitly grants pull-requests: write and issues: write.

Fix in Claude CodeFix in ConductorFix in CursorFix in Codex

@github-actions

Copy link
Copy Markdown
Contributor

E2E Test Results

All tests passed • 307 passed • 1 skipped • 1149s

StatusCount
✅ Passed307
❌ Failed0
⚠️ Flaky1
⏭️ Skipped1

Tests ran across 4 shards in parallel.

View full report →

@github-actions

Copy link
Copy Markdown
Contributor

Deep Review

A single-file change to .github/workflows/deep-review.yml that re-enables fork-PR reviews (allow-unsafe-pr-checkout: true) and hardens reviewer confinement (persist-credentials: false, --strict-mcp-config, a narrowed --allowedTools, and a new step materializing prior review comments). The confinement is largely sound: reviewers confirmed no fork code is executed after checkout (no install/build/test; the CLI install uses --ignore-scripts on a SHA-512-pinned tarball), git -c diff.external/git config exec is genuinely blocked by prefix matching, and .mcp.json/.claude/agents/.claude/skills are neutralized. The findings below concentrate on one confinement gap and one new failure mode.

🔴 P0/P1 -- must fix

  • .github/workflows/deep-review.yml:656 -- The confinement flags neutralize .mcp.json, .claude/agents, and .claude/skills, but a fork-supplied project-memory file (CLAUDE.md, .claude/CLAUDE.md) is not known to be suppressed by --setting-sources user and would load from the newly checked-out fork tree as unfenced, trusted instructions rather than fenced evidence.
    • Fix: Verify on the pinned CLI whether --setting-sources user suppresses project-memory loading, and if it does not, disable memory or run the reviewer from a directory outside the fork tree.
    • security, adversarial

🟡 P2 -- recommended

  • .github/workflows/deep-review.yml:271 -- The new Materialize prior review comments step has no continue-on-error, so a transient error in any of its three github.paginate calls fails the whole job and posts no review, converting the previous single-persona degradation into total review loss.
    • Fix: Add continue-on-error: true and have the script write the empty-state file and set count=0 on failure so the fan-out proceeds without the previous-comments persona.
    • correctness, reliability
  • .github/workflows/deep-review.yml:143 -- With the fork tree and a pull-requests: write token now coexisting in the job, an injected reviewer that can read files could surface an on-disk gh token into the posted review comment, since the credential scrub covers subprocess env but not files on disk.
    • Fix: Confirm where claude-code-action stores the gh token and ensure it lives outside any reviewer-readable path, or scope the token down.
🔵 P3 nitpicks (5)
  • .github/workflows/deep-review.yml:515 -- The prompt interpolates steps.prior.outputs.count and unconditionally tells the reviewer to read .deep-review/prior-comments.md, which renders ( found) and points at a missing file if the prior step is ever skipped or fails.
    • Fix: Guarantee the file always exists and default the count so the reviewer never chases a missing path.
  • .github/workflows/deep-review.yml:297 -- The three github.paginate calls have no retry/backoff and the step has no timeout-minutes, so a transient 5xx throws immediately and a hung request stalls until the job default.
    • Fix: Add a bounded retry with backoff and a short timeout-minutes on the step.
  • .github/workflows/deep-review.yml:289 -- The sanitizer, fence contract, truncation-marker format, and length caps are duplicated across the two github-script steps and already diverge, where the title strips \n/\t but clean() preserves them.
    • Fix: Extract the shared fence/sanitizer into one module used by both steps, or cross-link them so the isolation contract stays in lockstep.
  • .github/workflows/deep-review.yml:282 -- OURS hard-codes comment markers owned by claude-code-review.yml and deep-resolve.yml, so a marker rename in a sibling workflow silently breaks the self-comment filter and makes the persona re-report bot comments as unaddressed feedback.
    • Fix: Document per-marker ownership at the list so a sibling rename is caught in review.
  • .github/workflows/deep-review.yml:658 -- The enumerated git/gh allowlist is implicitly coupled to the externally-pinned plugin and its personas, so a PLUGIN_REF bump that needs a new subcommand degrades coverage silently via permission denial rather than a hard failure.
    • Fix: Note at the PLUGIN_REF definition that bumps require re-auditing this allowlist, and surface permission_denials_count in the sandbox health step.

Reviewers (6): correctness, security, reliability, adversarial, maintainability, project-standards.

Testing gaps:

  • No test that the prior-comments step degrades gracefully and the review still posts when the comments API fails.
  • No canary test that a fork-supplied CLAUDE.md / project-memory file does not influence reviewer output under --setting-sources user.
  • No test that a write-scoped token is never left on disk readable by the reviewer after both persist-credentials: false checkouts.

@teeohhem
teeohhem merged commit f3e07a7 into mainAug 21, 2026
31 checks passed
@teeohhem
teeohhem deleted the tom/trim-comments branch August 21, 2026 12:45
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/tier-1Trivial — auto-merge candidate once CI passes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@teeohhem