Validate the structured Copilot review marker on live reviews

[ptr727]

Summary

Validate the structured Copilot review marker introduced by #793 against live GitHub reviews, then make pr_review.py enforce every field that proves useful.

Context

#793 instructs Copilot code review to end each review body with:

<!-- fleet-review: reviewed=N changed=N findings=N -->

scripts/pr_review.py reads reviewed and changed as a stable alternative to Copilot's changing coverage prose. It retains the legacy prose readers for reviews created before the marker ships. An absent coverage statement blocks with exit 45.

The implementation deliberately does not treat one local parser test as evidence that GitHub-hosted Copilot follows the instruction. It also parses the findings field as part of the marker shape but does not yet compare that value with inline or body-only findings.

Measurement

After #793 is available on a pull request head:

1. Request Copilot review on at least one clean change and one change expected to produce a finding.
2. Record whether the review body carries exactly one marker and whether GitHub preserves the HTML comment.
3. Record whether reviewed and changed match the pull request file set.
4. Record how findings relates to inline threads, review-body findings, and any suppressed block.
5. Repeat after one fix push to verify the marker describes the current round rather than an earlier round.
6. Capture any alternate output shape verbatim in fixtures before changing the parser.

Implementation Scope

• Add corpus-derived fixtures for every observed marker shape.
• Fail closed on duplicate, malformed, impossible, or conflicting markers.
• Decide whether findings=N can be reconciled reliably with GitHub's review objects.
• If it can, report the declared and observed counts in status and block on disagreement.
• If it cannot, remove findings from the contract rather than retaining a field that appears enforced but is ignored.
• Confirm that the no-suppression instruction prevents hidden findings. Keep the existing suppressed-block reader as backward-compatible defense.
• Update the code-review skill, Copilot bootstrap, parser help, and tests together when the measured format differs.

Acceptance Criteria

• Evidence links identify the live review rounds used for validation.
• Tests reproduce the clean, finding, and post-fix rounds from sanitized review bodies.
• The documented marker exactly matches the parser's accepted shape.
• Every retained marker field affects a reported or blocking decision.
• Missing or malformed structured coverage cannot exit successfully.
• Legacy pre-marker coverage bodies remain readable until an explicit retirement decision is recorded.

Relationship to Existing Issues

#788 identified the unstated-coverage ambiguity. #793 introduces the structured marker and exit 45. This issue validates the external reviewer behavior that neither local unit tests nor CI can exercise.

[ptr727]

Live evidence from PR #799 review 4955381803:

• Copilot reported full coverage using legacy prose: Copilot reviewed 12 out of 12 changed files ....
• It did not emit the required <!-- fleet-review: reviewed=N changed=N findings=N --> marker.
• It emitted one inline finding and one Suppressed comments (1) block despite the explicit instruction to publish every supported finding and never suppress one.
• The review footer did not attribute the repository's .agents/skills/code-review skill and instead suggested adding .github/skills/code-review/SKILL.md.

This is the clean baseline showing that the bootstrap text alone does not enforce the schema or no-suppression contract. Re-run the same PR shape after #797 supplies a Copilot-discoverable skill.

[ptr727]

Second-round evidence from PR #799 review 4955414705, after fixing both first-round findings:

• The review covered the current f1c24d7 head and reported 12 of 12 files using legacy prose.
• It generated no new comments and no suppressed finding on this head.
• It again omitted the required fleet-review marker.
• It carried no skill attribution or footer indicating that .agents/skills/code-review loaded.

This removes findings as a possible reason the first marker was absent. The structured contract is not being followed on either a finding-bearing round or a clean round.

[ptr727]

Live evidence from PR #799 after .github/skills/ was generated on the head branch:

• da7249f review: legacy full coverage, no structured marker, two suppressed findings.
• c3738e5 review: legacy full coverage, no structured marker, one suppressed finding.
• 04f4bf7 review: legacy full coverage, no structured marker, one suppressed finding.
• ced266c review: legacy full coverage, no structured marker, no findings.

All four rounds reviewed 53 of 53 changed files. None emitted the required fleet-review marker or visible skill attribution. Three rounds continued to use hidden suppressed-finding blocks despite the code-review skill and always-on instructions explicitly forbidding suppression.

The findings demonstrate that Copilot inspected the generated skill files as changed content, but they do not prove that Copilot invoked the code-review skill. The missing attribution and marker leave skill activation unverified.

Keep this issue open as a behavior and observability tracker. The feature is generally available, but the live implementation still does not reliably honor or attest to the requested schema and no-suppression contract.

[ptr727]

Current audit against develop at 1832702: partial. Live rounds recorded here establish that Copilot omits the structured marker and can still suppress findings. The current code-review skill continues to require the marker, while pr_review.py retains legacy coverage parsing. No retained findings reconciliation or replacement contract resolves the observed external behavior.