What happened
On PR #176, human reviewer PatAKnight caught 5 design-level concerns about RHDH version resolution semantics, air-gap behavior, and CLI UX that the review agent missed entirely. The agent's successful review ($8.81, opus) produced only code-level findings — valid but lower-impact. The repo's AGENTS.md has empty ## Key Conventions, ## Architecture, and ## Pattern References sections (only HTML comment placeholders), leaving the review agent with no domain context about RHDH releases, Backstage version mapping, the CLI command structure, or air-gapped deployment requirements.
Specific human findings the agent missed:
- Falling back to
main when a release branch doesn't exist could silently resolve the wrong Backstage version
RHDH_OFFLINE only skips GitHub metadata — the Backstage manifest still fetches from versions.backstage.io, so this is not air-gapped- Unprefixed versions like
1.54.0 are silently treated as Backstage versions, which is easy to misread as RHDH
- The static compatibility matrix will require manual updates every release cycle
backstage:^ peer dependencies should show an explicit unverifiable message rather than silently flagging as invalid
What could go better
The review agent had no way to reason about domain-specific design trade-offs because the AGENTS.md provides only build/test commands and PR conventions — zero architectural context. The human reviewer's design-level comments required understanding: (1) RHDH release branches and their relationship to Backstage versions, (2) the distinction between offline metadata lookup and air-gapped manifest resolution, (3) version input disambiguation (RHDH vs Backstage), and (4) static compatibility matrix maintenance burden.
High confidence that populating these sections would improve agent review depth for feature PRs touching version resolution or CLI commands. Moderate confidence for other PR types — the agent may still miss nuances requiring deep product knowledge, but having architectural context establishes a baseline the agent currently lacks entirely.
Proposed change
Fill in the three empty AGENTS.md sections:
Key Conventions: Document the RHDH-to-Backstage version mapping model (release branches, static matrix fallback, backstage: prefix escape hatch), the offline vs air-gapped distinction (RHDH_OFFLINE skips GitHub metadata but not manifest fetch; --manifest-file is needed for true air-gap), and the convention that CLI commands set process.exitCode rather than calling process.exit() directly for testability.
Architecture: Document the version resolution engine (src/lib/rhdhVersion.ts) as the core abstraction — it resolves RHDH version aliases to Backstage release versions via GitHub metadata with a static matrix fallback. Document the command structure under src/commands/ (each command is a directory with command.ts exporting registration and handler). Note the manifest caching strategy (keyed by version + base URL).
Pattern References: Point to src/commands/check-versions/command.ts as the reference for new CLI commands, src/lib/rhdhVersion.ts for version resolution patterns, and src/lib/rhdhVersion.test.ts for test patterns using mocked fetch responses.
Validation criteria
On the next 3 feature PRs that touch version resolution or add a new CLI command, the review agent should surface at least one design-level finding related to RHDH domain semantics (e.g., version mapping correctness, offline/air-gap behavior, user-facing error messages) rather than only code-level syntax and style findings.
Generated by retro agent from #176
What happened
On PR #176, human reviewer PatAKnight caught 5 design-level concerns about RHDH version resolution semantics, air-gap behavior, and CLI UX that the review agent missed entirely. The agent's successful review ($8.81, opus) produced only code-level findings — valid but lower-impact. The repo's
AGENTS.mdhas empty## Key Conventions,## Architecture, and## Pattern Referencessections (only HTML comment placeholders), leaving the review agent with no domain context about RHDH releases, Backstage version mapping, the CLI command structure, or air-gapped deployment requirements.Specific human findings the agent missed:
mainwhen a release branch doesn't exist could silently resolve the wrong Backstage versionRHDH_OFFLINEonly skips GitHub metadata — the Backstage manifest still fetches fromversions.backstage.io, so this is not air-gapped1.54.0are silently treated as Backstage versions, which is easy to misread as RHDHbackstage:^peer dependencies should show an explicit unverifiable message rather than silently flagging as invalidWhat could go better
The review agent had no way to reason about domain-specific design trade-offs because the AGENTS.md provides only build/test commands and PR conventions — zero architectural context. The human reviewer's design-level comments required understanding: (1) RHDH release branches and their relationship to Backstage versions, (2) the distinction between offline metadata lookup and air-gapped manifest resolution, (3) version input disambiguation (RHDH vs Backstage), and (4) static compatibility matrix maintenance burden.
High confidence that populating these sections would improve agent review depth for feature PRs touching version resolution or CLI commands. Moderate confidence for other PR types — the agent may still miss nuances requiring deep product knowledge, but having architectural context establishes a baseline the agent currently lacks entirely.
Proposed change
Fill in the three empty
AGENTS.mdsections:Key Conventions: Document the RHDH-to-Backstage version mapping model (release branches, static matrix fallback,
backstage:prefix escape hatch), the offline vs air-gapped distinction (RHDH_OFFLINEskips GitHub metadata but not manifest fetch;--manifest-fileis needed for true air-gap), and the convention that CLI commands setprocess.exitCoderather than callingprocess.exit()directly for testability.Architecture: Document the version resolution engine (
src/lib/rhdhVersion.ts) as the core abstraction — it resolves RHDH version aliases to Backstage release versions via GitHub metadata with a static matrix fallback. Document the command structure undersrc/commands/(each command is a directory withcommand.tsexporting registration and handler). Note the manifest caching strategy (keyed by version + base URL).Pattern References: Point to
src/commands/check-versions/command.tsas the reference for new CLI commands,src/lib/rhdhVersion.tsfor version resolution patterns, andsrc/lib/rhdhVersion.test.tsfor test patterns using mocked fetch responses.Validation criteria
On the next 3 feature PRs that touch version resolution or add a new CLI command, the review agent should surface at least one design-level finding related to RHDH domain semantics (e.g., version mapping correctness, offline/air-gap behavior, user-facing error messages) rather than only code-level syntax and style findings.
Generated by retro agent from #176