chore(fullsend): use sonnet for reviews - #201
Conversation
|
|
🤖 Finished Review · ✅ Success · Started 2:49 PM UTC · Completed 2:58 PM UTC Commit: Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $1.52 |
|
Risk Assessment: low (1/5) DetailsSingle-line config-only change with no protected paths, no security-sensitive files, and no dependency changes; a minor regression history signal (two prior fix commits toggling the model on this same file) is noted but insufficient to elevate the score above low. |
|
Looks good to me |
|
|
|
🤖 Finished Retro · ✅ Success · Started 3:07 PM UTC · Completed 3:17 PM UTC Commit: Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $2.70 |
Retro: PR #201 —
|
| Time (UTC) | Event |
|---|---|
| 14:48:21 | kadel opens PR #201 (kadel-patch-1 → main) |
| 14:48:49 | Review agent starts (run 34491484232), opus / high effort |
| 14:50:09 | gashcrumb (human) approves (~2 min after PR creation) |
| 14:57:13 | Review agent completes (6m43s runtime, $1.52) — zero findings, approves |
| 14:58:13 | Bot posts APPROVED review |
| 15:01:15 | gashcrumb notes unrelated homepage CI failure (RHDHBUGS-3746) |
| 15:06:37 | PR merged |
Workflow quality
- Review quality: Correct outcome. The agent identified the change as trivial, constrained sub-agents to ≤5 tool calls each, skipped 4 of 7 sub-agents (security, intent-coherence, docs-currency, cross-repo-contracts), and approved with zero findings. Risk assessment (1/5) included a nuanced note about regression history on the same file.
- Rework rate: Zero — single review cycle, immediate approval from both human and bot.
- Autonomy: Human and bot reviewers agreed. Both approved without inline comments. The bot's review provided a useful risk-assessment signal but didn't surface anything the human needed to act on.
- Routing: Correct. Only the review agent (on PR open) and retro agent (on merge) were dispatched; all other stages were correctly skipped.
Cost observation
$1.52 on opus for a 1-line config change is disproportionate relative to the change scope, though the agent's trivial-change detection did constrain the work appropriately. This is self-correcting — the PR itself switches future reviews to sonnet, which will significantly reduce per-review cost.
Evidence for existing issues in fullsend-ai/agents
All improvement opportunities identified in this retro are already tracked by open issues:
- #301: Even after trivial classification, the review agent dispatched 3 sub-agents and took 6m43s. This PR provides evidence that the trivial path could be more lightweight.
- #879: This was a trivial human-authored PR (by a repo maintainer) that would benefit from the proposed fast-path.
- #497: Sub-agents ran on opus despite trivial scope classification. Model downgrade would have reduced cost.
- #1029: The review agent classified adding the new
modelkey as "value-only," but this is actually a new key introduction with behavioral implications (changes which AI model runs reviews). This matches the scenario described in #1029. - #649: A retro was triggered on a trivially-reviewed, zero-finding PR. This retro itself is evidence that the early-exit optimization described in #649 would save resources.
Conclusion
No new proposals — the workflow executed correctly, and all systematic improvement opportunities are already covered by existing issues in the agents repo.
sonnet should work now, it was enabled on gcp