Skip to content

chore: sync workflow templates - #284

Closed
stranske wants to merge 1 commit into
mainfrom
sync/workflows-f0578787979a
Closed

stranske wants to merge 1 commit into
mainfrom
sync/workflows-f0578787979a

Conversation

@stranske

Copy link
Copy Markdown
Owner

Sync Summary

Files Updated

  • agents-81-gate-followups.yml: Gate followups hub - consolidates keepalive and autofix followups
  • agents-bot-comment-handler.yml: Bot comment handler - dispatches agents to address bot review comments (deprecated; replaced by agents-80-pr-event-hub.yml, removal no earlier than 2026-02-15)
  • agents-weekly-metrics.yml: Weekly metrics - aggregates auto-pilot, keepalive, autofix and verifier metrics into summary reports
  • aggregate_agent_metrics.py: Aggregates downloaded weekly agent metrics - required by agents-weekly-metrics.yml
  • terminal_disposition.js: Machine-readable terminal disposition records and source summaries
  • terminal_disposition_coverage.js: Warning-only terminal disposition source coverage preflight
  • bot_comment_auth_coverage.js: Warning-only bot-comment App auth coverage preflight
  • coverage_monitor_summary.js: Machine-readable weekly coverage monitor checkpoint
  • weekly_metrics_artifacts.js: Bounded weekly metrics artifact selection contract

Files Skipped

  • pr-00-gate.yml: File exists and sync_mode is create_only
  • ci.yml: File exists and sync_mode is create_only
  • dependabot.yml: File exists and sync_mode is create_only
  • llm_slots.json: None

Review Checklist

  • CI passes with updated workflows
  • No repo-specific customizations were overwritten

Source: stranske/Workflows
Manifest: .github/sync-manifest.yml

Automated sync from stranske/Workflows
Template hash: f0578787979a

Changes synced from sync-manifest.yml
@stranske stranske added sync Automated sync from Workflows automated Automated sync from Workflows labels Apr 26, 2026
Copilot AI review requested due to automatic review settings April 26, 2026 06:14
@stranske stranske added the automated Automated sync from Workflows label Apr 26, 2026
@stranske-keepalive

stranske-keepalive Bot commented Apr 26, 2026

Copy link
Copy Markdown
Contributor

🤖 Keepalive Loop Status

PR #284 | Agent: Codex | Iteration 0/5

Current State

Metric Value
Iteration progress [----------] 0/5
Action wait (missing-agent-label)
Disposition skipped (transient)
Gate success
Tasks 0/13 complete
Timeout 45 min (default)
Timeout usage 4m elapsed (11%, 41m remaining)
Keepalive ❌ disabled
Autofix ❌ disabled

🔍 Failure Classification

| Error type | infrastructure |
| Error category | resource |
| Suggested recovery | Confirm the referenced resource exists (repo, PR, branch, workflow, or file). |

@stranske-keepalive

stranske-keepalive Bot commented Apr 26, 2026

Copy link
Copy Markdown
Contributor
Keepalive Work Log (click to expand)
# Time (UTC) Agent Action Result Files Tasks Progress Commit Gate
0 2026-04-26 06:16:29 Codex wait (missing-agent-label-transient) skipped 0 0/13
0 2026-04-26 06:18:45 Codex wait (missing-agent-label-transient) skipped 0 0/13 cancelled
0 2026-04-26 06:19:23 Codex wait (missing-agent-label-transient) skipped 0 0/13 success

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Syncs workflow templates and supporting scripts to improve weekly agent metrics reporting and coverage/preflight observability across workflows.

Changes:

  • Extend aggregate_agent_metrics.py to emit a JSON “summary contract”, enrich verifier/model stats, and include detailed parse-error reporting.
  • Add/propagate new telemetry and reporting fields across weekly metrics artifact selection, terminal disposition coverage, and bot-comment auth coverage.
  • Update workflows to upload new JSON artifacts and emit wrapper terminal-disposition records for bot-comment handler runs.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
scripts/aggregate_agent_metrics.py Adds parse-error detail tracking, verifier model metadata reporting, and emits a JSON summary contract alongside markdown.
.github/workflows/agents-weekly-metrics.yml Publishes the new JSON weekly metrics summary artifact.
.github/workflows/agents-bot-comment-handler.yml Adds skip_reason output and emits/upload wrapper terminal disposition telemetry.
.github/workflows/agents-81-gate-followups.yml Adjusts jq invocation to compact output.
.github/scripts/weekly_metrics_artifacts.js Reports missing/selected priority families and exposes priority family status details.
.github/scripts/terminal_disposition_coverage.js Adds verifier model compatibility checks and richer artifact-selection/status normalization for terminal disposition coverage.
.github/scripts/terminal_disposition.js Extends terminal disposition schema normalization with model/reason/verifier_mode fields.
.github/scripts/coverage_monitor_summary.js New script to aggregate coverage monitor reports into a single JSON/MD contract.
.github/scripts/bot_comment_auth_coverage.js Tracks wrapper decisions about reusable invocation and improves organic evidence requirement reporting.


- name: Upload wrapper terminal disposition
if: always()
uses: actions/upload-artifact@v7

Copilot AI Apr 26, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actions/upload-artifact is referenced with the floating @v7 tag here, while the rest of the repo pins it to a commit SHA (e.g. @043fb46... # v7). Please pin this action to a specific SHA to match repo convention and avoid supply-chain risk.

Suggested change
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@043fb46c9edb0b0c7c6e8f6f0b0d3f0a4f7a6c52 # v7

Copilot uses AI. Check for mistakes.
Comment on lines +384 to +387
model_text = str(model)
verifier_models[model_text] += 1
if model_text.lower() in unsupported_models:
unsupported_verifier_models[model_text] += 1

Copilot AI Apr 26, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Model names are matched against unsupported_models using model_text.lower(), but the counters use the original model_text as the key. This can split counts across different casings (and also make the unsupported model counters not line up cleanly with the normalized unsupported list). Consider normalizing the counter keys consistently (e.g., lowercasing once) before incrementing.

Suggested change
model_text = str(model)
verifier_models[model_text] += 1
if model_text.lower() in unsupported_models:
unsupported_verifier_models[model_text] += 1
normalized_model_text = str(model).lower()
verifier_models[normalized_model_text] += 1
if normalized_model_text in unsupported_models:
unsupported_verifier_models[normalized_model_text] += 1

Copilot uses AI. Check for mistakes.
Comment on lines +568 to +572
"by_artifact_family": dict(sorted(family_counts.items())),
"by_artifact": dict(sorted(artifact_counts.items())),
"by_reason": dict(sorted(reason_counts.items())),
"details": [detail.as_dict() for detail in parse_error_details],
}

Copilot AI Apr 26, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_parse_error_contract includes a details array with an entry per parse error. For a badly corrupted or very large input, this can make the JSON summary artifact extremely large. Consider bounding the exported details (similar to the markdown table cap) and/or adding an env flag to include full details only when needed, while keeping the aggregate counts always.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated Automated sync from Workflows sync Automated sync from Workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants