Skip to content

Add Claude Code workflow for AI-assisted PR reviews - #4738

Draft
shreyas-goenka wants to merge 6 commits into
mainfrom
add-claude-code-workflow
Draft

Add Claude Code workflow for AI-assisted PR reviews#4738
shreyas-goenka wants to merge 6 commits into
mainfrom
add-claude-code-workflow

Conversation

@shreyas-goenka

@shreyas-goenkashreyas-goenka commented Mar 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a GitHub Actions workflow for AI-assisted PR reviews and interactive @claude mentions. This is a thin dispatcher — it triggers execution in databricks-eng/eng-dev-ecosystem on protected runners via the DECO workflow trigger GitHub App.

  • Review: automatic on PR open
  • Assist: triggered by @claude comments, can edit and push

Access restricted to org MEMBER/OWNER via author_association allowlists.

Depends on

https://github.com/databricks-eng/eng-dev-ecosystem/pull/1202

Test plan

  • End-to-end: review posted on CLI PR via dispatched workflow
  • End-to-end: @claude assist mode tested

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Mar 13, 2026

Copy link
Copy Markdown
Collaborator

Commit: 3cf6d9b

Run: 23075285472

Env🔄​flaky💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
💚​aws linux872687876:21
💚​aws windows872707854:47
🔄​aws-ucws linux2773647027:40
🔄​aws-ucws windows2773667006:31
💚​azure linux292717857:07
💚​azure windows292737837:35
🔄​azure-ucws linux4193676988:21
🔄​azure-ucws windows2193716966:41
💚​gcp linux292677886:15
💚​gcp windows292697864:52
18 interesting tests: 7 SKIP, 6 RECOVERED, 5 flaky
Test Nameaws linuxaws windowsaws-ucws linuxaws-ucws windowsazure linuxazure windowsazure-ucws linuxazure-ucws windowsgcp linuxgcp windows
🔄​TestAccept💚​R💚​R🔄​f💚​R💚​R💚​R💚​R🔄​f💚​R💚​R
🙈​TestAccept/bundle/resources/permissions🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions💚​R💚​R💚​R💚​R🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct💚​R💚​R💚​R💚​R
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform💚​R💚​R💚​R💚​R
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions💚​R💚​R💚​R💚​R🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct💚​R💚​R💚​R💚​R
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform💚​R💚​R💚​R💚​R
🙈​TestAccept/bundle/resources/postgres_branches/basic🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_branches/recreate🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_branches/update_protected🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_branches/without_branch_id🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_endpoints/recreate🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/synced_database_tables/basic🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🔄​TestAccept/ssh/connect-serverless-gpu🙈​s🙈​s🔄​f🔄​f🙈​s🙈​s🔄​f🔄​f🙈​s🙈​s
🔄​TestAccept/ssh/connection💚​R💚​R💚​R🔄​f💚​R💚​R🔄​f💚​R💚​R💚​R
🔄​TestFsLsWithAbsolutePaths✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p
🔄​TestFsLsWithAbsolutePaths/uc-volumes🙈​s🙈​s✅​p✅​p🙈​s🙈​s🔄​f✅​p🙈​s🙈​s
Top 20 slowest tests (at least 2 minutes):
durationenvtestname
5:35azure windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:45azure linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:16azure linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:08gcp linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:47azure windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:43gcp linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:13gcp windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:11aws-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:07gcp windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:58aws-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:57aws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:50aws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:44aws-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:43aws-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:40aws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:39aws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:18azure-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:11azure-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:09azure-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:03azure-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

@shreyas-goenka
shreyas-goenkaforce-pushed the add-claude-code-workflow branch from b418b20 to 0d2b698CompareMarch 13, 2026 19:56
@shreyas-goenka
shreyas-goenkaforce-pushed the add-claude-code-workflow branch from 0d2b698 to 6dbf8a3CompareMarch 13, 2026 20:03
@shreyas-goenka
shreyas-goenkaforce-pushed the add-claude-code-workflow branch from 6dbf8a3 to 275b67aCompareMarch 13, 2026 20:17
@shreyas-goenka
shreyas-goenkaforce-pushed the add-claude-code-workflow branch from 275b67a to fc25d50CompareMarch 13, 2026 20:25
@shreyas-goenka
shreyas-goenkaforce-pushed the add-claude-code-workflow branch from fc25d50 to 47a7034CompareMarch 13, 2026 20:33
Add a GitHub Actions workflow that provides AI-assisted PR reviews
and interactive @claude mentions using Claude Code backed by
Databricks Model Serving.
The workflow dispatches to eng-dev-ecosystem's protected runners
(whose IPs are allowlisted by the Databricks account IP ACL) via
the DECO workflow trigger GitHub App. Two modes:
- Review: automatic on PR open, posts a review comment
- Assist: triggered by @claude mentions, can edit code and push
Access is restricted to COLLABORATOR/MEMBER/OWNER via
author_association allowlists.
Co-authored-by: Isaac

@eng-dev-ecosystem-boteng-dev-ecosystem-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Workflow looks well-structured overall. Good security practices: fork PR checks, author_association allowlists, and using process.env to avoid script injection for comment_body. Two minor observations below.

Comment thread.github/workflows/claude-code.yml Outdated
workflow_id: 'cli-claude-code.yml',
ref: 'main',
inputs: {
pull_request_number: '${{ steps.pr.outputs.number }}',

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: steps.pr.outputs.number is interpolated directly into the script block via ${{ }}. This is safe in practice (it's a GitHub-assigned integer), but it's inconsistent with line 131 where comment_body correctly uses process.env to avoid expression injection.

For defense-in-depth, consider passing this through an env var too:

env:
COMMENT_BODY: ${{ github.event.comment.body }}PR_NUMBER: ${{ steps.pr.outputs.number }}
pull_request_number: process.env.PR_NUMBER,

Comment thread.github/workflows/claude-code.yml Outdated
(github.event_name == 'issue_comment' && github.event.issue.pull_request && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude'))
)
runs-on:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: The assist job has no concurrency group, unlike review (line 34-36). Multiple @claude mentions on the same PR could trigger parallel dispatches. If that's intentional, a brief comment would help; otherwise, consider adding:

concurrency:
group: claude-assist-${{ github.event.issue.number || github.event.pull_request.number }}cancel-in-progress: true

@shreyas-goenka

Copy link
Copy Markdown
ContributorAuthor

@claude reply to me in this PR with a comment saying hi

@shreyas-goenka

Copy link
Copy Markdown
ContributorAuthor

@claude say hi

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Mar 16, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: f2eb6a2

Run: 27830298355

Env❌​FAIL🔄​flaky💚​RECOVERED🙈​SKIP✅​pass🙈​skipTime
💚​aws linux71326510115:27
💚​aws windows71326710095:58
💚​aws-ucws linux7133619255:59
💚​aws-ucws windows7133639237:00
💚​azure linux11526810095:21
💚​azure windows11527010078:16
🔄​azure-ucws linux31153639216:53
❌​azure-ucws windows2111536591910:53
💚​gcp linux11526410125:37
💚​gcp windows11526610108:19
26 interesting tests: 13 SKIP, 7 RECOVERED, 4 flaky, 2 FAIL
Test Nameaws linuxaws windowsaws-ucws linuxaws-ucws windowsazure linuxazure windowsazure-ucws linuxazure-ucws windowsgcp linuxgcp windows
💚​TestAccept💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R💚​R
🙈​TestAccept/bundle/invariant/no_drift🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/permissions🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions💚​R💚​R💚​R💚​R🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct💚​R💚​R💚​R💚​R
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform💚​R💚​R💚​R💚​R
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions💚​R💚​R💚​R💚​R🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct💚​R💚​R💚​R💚​R
💚​TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform💚​R💚​R💚​R💚​R
🙈​TestAccept/bundle/resources/postgres_branches/basic🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_branches/recreate🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_branches/replace_existing🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_branches/update_protected🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_branches/without_branch_id🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_endpoints/basic🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/postgres_projects/update_display_name🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/synced_database_tables/basic🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🙈​TestAccept/ssh/connection🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S🙈​S
🔄​TestFsCpFileToFileFileNotOverwritten/dbfs_to_dbfs✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p
🔄​TestFsCpFileToFileFileNotOverwritten/local_to_dbfs✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p✅​p
🔄​TestFsCpFileToFileFileNotOverwritten/local_to_uc-volumes🙈​s🙈​s✅​p✅​p🙈​s🙈​s🔄​f✅​p🙈​s🙈​s
❌​TestFetchRepositoryInfoAPI_FromRepo✅​p✅​p✅​p✅​p✅​p✅​p✅​p❌​F✅​p✅​p
❌​TestFetchRepositoryInfoAPI_FromRepo/root✅​p✅​p✅​p✅​p✅​p✅​p✅​p❌​F✅​p✅​p
🔄​TestFetchRepositoryInfoAPI_FromRepo/subdir✅​p✅​p✅​p✅​p✅​p✅​p✅​p🔄​f✅​p✅​p
Top 20 slowest tests (at least 2 minutes):
durationenvtestname
4:48gcp windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:17gcp windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:14gcp linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:10gcp linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:19aws-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:06azure windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:02aws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:56aws-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:53aws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:52azure linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:51aws-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:48azure-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:47aws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:44azure windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:44azure-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:39aws-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:35aws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:35azure linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:33azure-ucws linuxTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:31azure-ucws windowsTestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

@eng-dev-ecosystem-boteng-dev-ecosystem-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean, well-structured workflow. Good security practices: fork checks on both jobs, author association gating, and passing comment_body through process.env instead of ${{ }} interpolation to avoid script injection. One suggestion below.

Comment thread.github/workflows/claude-code.yml Outdated
(
(github.event_name == 'issue_comment' && github.event.issue.pull_request && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude'))
)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: The review job has a concurrency group to cancel in-progress runs, but assist does not. If someone posts multiple @claude comments in quick succession, they could all trigger simultaneously. Consider adding:

concurrency:
group: claude-assist-${{ github.event.issue.number || github.event.pull_request.number }}cancel-in-progress: true

@shreyas-goenka

Copy link
Copy Markdown
ContributorAuthor

@claude is elon musk the founder of computing?

After dispatching to eng-dev-ecosystem, the workflow now finds the
remote run, posts a link in the step summary, and waits for it to
complete. This keeps the check visible on the PR while Claude works.
Co-authored-by: Isaac
- Pin actions to commit SHAs instead of mutable version tags
- Add concurrency group to assist job to prevent parallel dispatches
- Pass PR number through env var for defense-in-depth against injection
Co-authored-by: Isaac
Reflects current implementation: composite action, standalone workflows,
OIDC policies, SHA-pinned actions, concurrency groups, allowed tools,
and comprehensive threat model.
Co-authored-by: Isaac
Co-authored-by: Isaac
@github-actions

Copy link
Copy Markdown
Contributor

This PR has not received an update in a while. If you want to keep this PR open, please leave a comment below or push a new commit and auto-close will be canceled.

Drop the interactive @claude assist job and the issue_comment /
pull_request_review_comment triggers; keep only the automatic review on
PR open. Also stop passing event_type to cli-claude-code.yml, which is
now review-only.
Co-authored-by: Isaac
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@shreyas-goenka@eng-dev-ecosystem-bot@simonfaltum