Skip to content

Add workflow run analysis for security-focused agentic workflows - #10952

Closed
mnkiefer with Copilot wants to merge 4 commits into
mainfrom
copilot/analyze-agentic-workflow-runs
Closed

Add workflow run analysis for security-focused agentic workflows#10952
mnkiefer with Copilot wants to merge 4 commits into
mainfrom
copilot/analyze-agentic-workflow-runs

Conversation

CopilotAI commented Jan 21, 2026

Copy link
Copy Markdown
Contributor

Analysis of three security workflows (code-scanning-fixer, security-fix-pr, security-review) reveals Code Scanning Fixer experiencing 40% failure rate while other workflows operate nominally.

Deliverables

Reports Generated

  • workflow-runs-analysis.html - Interactive report with metrics, run history, and visual indicators
  • workflow-runs-analysis.md - Markdown summary for quick reference

Analysis Results

Code Scanning Fixer ⚠️

  • 60% success rate (last 10 runs)
  • Failures at step 24: "Execute GitHub Copilot CLI"
  • Pattern: systematic failures every 2-3 runs
  • Recommended actions:
    • Increase timeout 20→25 minutes
    • Add retry logic for transient failures
    • Investigate MCP gateway connectivity

Security Fix PR ✅

  • 100% success rate (last 10 runs)
  • 419 total runs, no recent failures
  • Processes up to 5 alerts per run via autofix API

Security Review ℹ️

  • Event-driven (slash command trigger)
  • 90% skip rate (expected behavior)
  • 1 of 10 runs required action (active usage)

Workflow Comparison

WorkflowTriggerSuccess RateMax Fixes/Run
code-scanning-fixerEvery 30m60%1 alert
security-fix-prEvery 4h100%5 alerts
security-review/security-reviewN/A10 comments

Data Source

GitHub Actions API via MCP Server analyzing last 10-20 runs per workflow with job-level failure diagnosis.

Original prompt

Analyze and report on latest agentic workflow runs of:
code-scanning-fixer, security-fix-pr, security-review


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

CopilotAIand others added 3 commits January 21, 2026 06:58
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
CopilotAI changed the title [WIP] Analyze latest agentic workflow runs for reportingAdd workflow run analysis for security-focused agentic workflowsJan 21, 2026
CopilotAI requested a review from mnkieferJanuary 21, 2026 07:05
@mnkiefer

Copy link
Copy Markdown
Collaborator
  • Done analysis

@github-actions
github-actionsBot deleted the copilot/analyze-agentic-workflow-runs branch April 2, 2026 12:08
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@mnkiefer