## Why
PR #897 addressed issue #895 but verification identified concerns (verdict: **Unknown**). This follow-up addresses the remaining gaps with improved task structure to ensure explicit workflow steps, robust testing, and comprehensive documentation.
## Tasks
- [ ] Update the `agents-auto-pilot.yml` to include explicit, well-named steps that run both the metrics collector and step timer utilities using concrete command invocations.
- [ ] Enhance the test suite for the metrics collector to check that its output is generated in both the CLI and a designated log file, and that all output fields conform to `AUTOPILOT_METRICS_SCHEMA.md`.
- [ ] Implement tests to simulate failure scenarios in both the metrics collector and timer utilities, verifying that non-zero exit codes, step names, error categories, and environment details are correctly logged in the summary metrics report.
- [ ] Extend `AUTOPILOT_METRICS_SCHEMA.md` to comprehensively list and describe all output fields (including edge cases) produced by the metrics collector and to reflect any new behavior introduced in `keepalive-runner.js` and `autopilot_metrics_collector.py`.
- [ ] Review and update `keepalive-runner.js` to validate that token/dispatch handling aligns with the functional changes. Add unit tests that verify the correct behavior with different token and dispatch scenarios.
- [ ] Integrate file I/O and environment variable handling tests for `scripts/autopilot_metrics_collector.py` to capture and validate behavior across typical and edge-case execution environments.
## Acceptance Criteria
- [ ] The `agents-auto-pilot.yml` workflow includes explicit steps that execute the metrics collector script using `python scripts/autopilot_metrics_collector.py` and the step timer utility, with each step clearly named.
- [ ] The metrics collector script outputs a summary metrics report to both a designated log file and the CLI, with fields matching `AUTOPILOT_METRICS_SCHEMA.md`.
- [ ] Failure scenarios in the metrics collector and timer utilities correctly log non-zero exit codes, step names, error categories, and environment details in the summary metrics report.
- [ ] The `AUTOPILOT_METRICS_SCHEMA.md` file comprehensively lists and describes all output fields, including edge cases, and reflects new behavior from `keepalive-runner.js` and `autopilot_metrics_collector.py`.
- [ ] `keepalive-runner.js` validates token/dispatch handling with unit tests verifying correct behavior across different scenarios.
- [ ] File I/O and environment variable handling in `scripts/autopilot_metrics_collector.py` are tested across typical and edge-case environments.
## Implementation Notes
- Modify `agents-auto-pilot.yml` to ensure steps are explicit and verifiable in CI logs.
- Update `tests/test_metrics_collector.py` to cover CLI and log file outputs.
- Create `tests/test_failure_handling.py` for failure scenario testing.
- Revise `AUTOPILOT_METRICS_SCHEMA.md` to align with code and test changes.
- Adjust `keepalive-runner.js` and add `tests/test_keepalive_runner.js` for token/dispatch validation.
- Develop `tests/test_file_io_and_env_handling.py` for comprehensive environment testing.
<details>
<summary>Background (previous attempt context)</summary>
- Reliance on a GitHub compare link (main...codex/issue-895) for acceptance criteria reference failed because it lacked explicit, in-repo acceptance criteria, making verification ambiguous and non-testable. Instead, define and document clear, measurable, and testable in-repo acceptance criteria that detail the expected inputs, outputs, and behaviors of each component.
</details>