Conversation
Automated sync from stranske/Workflows Template hash: 9ec595328ed8 Changes synced from sync-manifest.yml
🤖 Keepalive Loop StatusPR #267 | Agent: Codex | Iteration 0/5 Current State
🔍 Failure Classification| Error type | infrastructure | |
Keepalive Work Log (click to expand)
|
There was a problem hiding this comment.
Pull request overview
Syncs in the latest weekly agent-metrics workflow templates, adding bot-comment auth coverage reporting and updating the weekly aggregation pipeline.
Changes:
- Adds
scripts/aggregate_agent_metrics.pyto aggregate downloaded NDJSON metrics into a markdown summary. - Extends the weekly metrics workflow to run a new bot-comment auth coverage preflight and honor both coverage hard-blocks.
- Updates weekly artifact selection logic to recognize the new bot-comment auth coverage artifact families.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
scripts/aggregate_agent_metrics.py |
New Python aggregator that buckets NDJSON metrics and emits a markdown summary for the weekly report. |
.github/workflows/agents-weekly-metrics.yml |
Updates the scheduled workflow to set up Node via setup-node, run bot-comment auth coverage preflight, upload new artifacts, and enforce combined hard-block logic. |
.github/scripts/weekly_metrics_artifacts.js |
Expands artifact-family detection to include bot-comment auth coverage artifacts via regex patterns and adds them to priority selection. |
.github/scripts/bot_comment_auth_coverage.js |
New script to scan downloaded artifacts for auth coverage JSON, summarize/enforce policy, and emit JSON+markdown reports. |
| files = _gather_metrics_files(metrics_paths, metrics_dir) | ||
| if not files: | ||
| print("No metrics files found to aggregate.", file=sys.stderr) | ||
| return 1 | ||
|
|
There was a problem hiding this comment.
main() returns exit code 1 when no .ndjson files are found. In the agents-weekly-metrics workflow, the download step explicitly tolerates having no artifacts, so this will cause the scheduled workflow to fail on weeks with no metrics. Consider writing a minimal markdown summary (e.g., with a generated timestamp and “no records found”) and returning 0, or gate the non-zero exit behind an opt-in env var (e.g., FAIL_ON_EMPTY=1).
| f"- Issues: {autopilot['issues']}", | ||
| f"- Total step executions: {autopilot['total_steps']}", | ||
| f"- Escalations: {autopilot['escalation_count']}", | ||
| f"- Needs-human rate: {_format_rate(autopilot['needs_human_count'], autopilot['issues'] or 1)}", |
There was a problem hiding this comment.
The needs-human rate denominator uses autopilot['issues'] or 1, which can silently report an incorrect percentage when there are auto-pilot records but no issue numbers were captured (issues=0). Since _format_rate() already handles 0/negative denominators by returning n/a, pass the actual issues count instead of forcing the denominator to 1.
| f"- Needs-human rate: {_format_rate(autopilot['needs_human_count'], autopilot['issues'] or 1)}", | |
| f"- Needs-human rate: {_format_rate(autopilot['needs_human_count'], autopilot['issues'])}", |
|
Closing as stale: newer sync workflow templates PR #339 is open for this repo. |
Sync Summary
Files Updated
Files Skipped
Review Checklist
Source: stranske/Workflows
Manifest:
.github/sync-manifest.yml