Uh oh!
There was an error while loading. Please reload this page.
chore(ci): remove LLM summary generation from vulnerability triage - #1334
Merged
Conversation
Replace the Claude analysis step in the vulnerability-triage workflow with deterministic jq/curl scripting. Findings are now built directly from the normalized Trivy, Dependabot, and CodeQL scan files (dedup by pre-computed id, CodeQL grouped by rule, templated titles/descriptions), and existing Linear issues are matched via GraphQL. No LLM is used anywhere. Drops the ANTHROPIC_API_KEY secret from the reusable workflow (no callers pass it). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
@brendan-kellam your pull request is missing a changelog! |
Contributor
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughThe vulnerability triage GitHub Actions workflow removes its dependency on Claude/Anthropic by replacing the AI analysis step with a deterministic jq pipeline. The workflow interface drops the ChangesVulnerability Triage Workflow Refactor
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
.github/workflows/vulnerability-triage.yml (1)
574-588: ⚡ Quick winConsider validating the Linear API response before processing.
If the GraphQL request fails (network error, rate limit, server error),
$RESPONSEwon't contain.data.issues.nodes, and the null-safe jq will produce{linearIssueExists: false, ...}. This would cause the workflow to create a duplicate issue when one actually exists but the lookup failed.♻️ Proposed fix to check for API errors
PAYLOAD=$(jq -n --arg query "$SEARCH_QUERY" --argjson vars "$VARS" '{query: $query, variables: $vars}') - RESPONSE=$(curl -s -X POST https://api.linear.app/graphql \+ HTTP_CODE=$(curl -s -o /tmp/linear-search.json -w "%{http_code}" -X POST https://api.linear.app/graphql \ -H "Content-Type: application/json" \ -H "Authorization: $LINEAR_API_KEY" \ -d "$PAYLOAD") + RESPONSE=$(cat /tmp/linear-search.json)++ if [ "$HTTP_CODE" != "200" ] || echo "$RESPONSE" | jq -e '.errors' >/dev/null 2>&1; then+ echo "::warning::Linear API error for $CVE_ID (HTTP $HTTP_CODE). Assuming no existing issue."+ fi SELECTED=$(echo "$RESPONSE" | jq --arg prefix "[$REPOSITORY]" '🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/vulnerability-triage.yml around lines 574 - 588, Add validation of the Linear API response in the vulnerability-triage.yml workflow before processing the jq filter on $RESPONSE. Check if the GraphQL response contains errors (using jq to detect .errors field) or if the expected data structure is missing, and fail the workflow step explicitly when the API call fails rather than silently treating a failed request as "issue not found". This prevents the workflow from incorrectly creating duplicate issues when the API lookup fails due to network errors, rate limits, or server errors.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In @.github/workflows/vulnerability-triage.yml:
- Around line 574-588: Add validation of the Linear API response in the
vulnerability-triage.yml workflow before processing the jq filter on $RESPONSE.
Check if the GraphQL response contains errors (using jq to detect .errors field)
or if the expected data structure is missing, and fail the workflow step
explicitly when the API call fails rather than silently treating a failed
request as "issue not found". This prevents the workflow from incorrectly
creating duplicate issues when the API lookup fails due to network errors, rate
limits, or server errors.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 69cf1786-392c-4626-bc42-bd0de18717ec
📒 Files selected for processing (1)
.github/workflows/vulnerability-triage.yml
Trivy can report the same CVE multiple times (across lockfiles/targets or when a CVE affects several packages). The previous build mapped over every Trivy entry, so duplicate ids produced duplicate findings and therefore duplicate Linear issues. Group Trivy and Dependabot by id so each unique id yields exactly one finding, collecting all affected packages into it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ates The Linear search bound the team UUID as a `String!` GraphQL variable into `team.id.eq`, which expects `ID`. That variable-type mismatch made every search return null data, so no finding ever matched an existing issue and the job re-filed every CVE on each run. Drop the team filter (repo-prefix title scoping is the authoritative filter anyway) and warn when a search errors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reworks
.github/workflows/vulnerability-triage.ymlso it no longer generates any LLM summary. The Claude analysis step is replaced with deterministicjq/curlscripting that reproduces everything Claude was doing:Build findings— a singlejqprogram builds thecveslist from the three normalized scan files:id(a CVE/GHSA in both Trivy and Dependabot merges into onetrivy+dependabotentry).idinto one entry listing every location.Match existing Linear issues— resolves the team UUID /CVElabel /Triagestate / API-key owner once, then searches Linear per finding, applies the[<repository>]title-prefix scoping, and prefers an open issue over a closed one.The downstream
Write findings summaryandCreate Linear issuessteps are unchanged in logic — they readfindings.json(same JSON shape the oldstructured_outputproduced) instead of the Claude step output, and the create step reuses the metadata resolved by the match step.Also:
ANTHROPIC_API_KEYsecret from the reusableworkflow_call(no callers pass it).Claude Analysis & Linear Triage→Linear Triage.Testing
Validated the
jqbuild program against sample data (Trivy↔Dependabot id collision merges, Dependabot-only retained, multi-location CodeQL grouped into one), the Linear match-selection logic (open repo-scoped preferred, other repos ignored, closed-only falls into the reopen path, no-match →exists:false), and the empty-findings edge case. YAML parses cleanly.Note: no CHANGELOG entry — this is an internal CI/security-automation change, not user-facing.
🤖 Generated with Claude Code
Summary by CodeRabbit