Conversation
Automated sync from stranske/Workflows Template hash: 050e85f549f0 Changes synced from sync-manifest.yml
There was a problem hiding this comment.
Pull request overview
Syncs workflow-consumer template scripts to support a new “codex-cli-freshness” weekly metrics artifact and include CLI version metadata in terminal disposition records.
Changes:
- Add
codex-cli-freshnessartifact family support to metrics artifact selection and aggregation. - Extend agent metrics aggregation to classify and summarize
workflows-codex-cli-freshness/v1records. - Include
llm_cli_versionin normalized terminal disposition records and surface CLI versions in verifier summaries.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
scripts/aggregate_agent_metrics.py |
Adds artifact family recognition, entry classification, verifier CLI version summarization, and a new Codex CLI freshness summary section/contract output. |
.github/scripts/weekly_metrics_artifacts.js |
Updates artifact-family allowlists/priorities to ensure codex CLI freshness artifacts are selected. |
.github/scripts/terminal_disposition.js |
Adds llm_cli_version field to the normalized terminal disposition payload. |
| max_patch_delta = 0 | ||
| update_targets = Counter() | ||
| for entry in entries: | ||
| status = str(entry.get("status") or "unknown") |
There was a problem hiding this comment.
In _summarise_codex_cli_freshness, status is counted using the raw string (str(entry.get('status') ...)) without trimming/normalizing, but later outdated_records / latest_unavailable_records are derived by looking up exact keys ("outdated", "latest-unavailable"). If the incoming data ever contains values like "Outdated" or trailing whitespace, these derived counts will be incorrect. Consider normalizing status (e.g., strip().lower()) before counting, and optionally mapping/validating to a small set of allowed status tokens.
| status = str(entry.get("status") or "unknown") | |
| status = str(entry.get("status") or "unknown").strip().lower() |
| dispatch_outcome: input.dispatch_outcome ?? input.dispatchOutcome, | ||
| llm_model: input.llm_model ?? input.llmModel ?? input.model, | ||
| model_selection_reason: input.model_selection_reason ?? input.modelSelectionReason, | ||
| llm_cli_version: input.llm_cli_version ?? input.llmCliVersion ?? input.cli_version, |
There was a problem hiding this comment.
llm_cli_version normalization includes input.cli_version but not a camelCase variant (e.g. input.cliVersion), while nearby fields consistently accept both snake_case and camelCase (e.g. run_id ?? runId). For consistency and to avoid silently dropping values from camelCase callers, consider including the camelCase fallback here too.
| llm_cli_version: input.llm_cli_version ?? input.llmCliVersion ?? input.cli_version, | |
| llm_cli_version: | |
| input.llm_cli_version ?? input.llmCliVersion ?? input.cli_version ?? input.cliVersion, |
Sync Summary
Files Updated
Files Skipped
Review Checklist
Source: stranske/Workflows
Source SHA:
8993a5697b5d28fa7e6afc45e7eb46bcb2ad5d5fTemplate hash:
050e85f549f0Sync branch:
sync/workflows-050e85f549f0Consumer repo:
stranske/TemplateManifest:
.github/sync-manifest.yml