Skip to content

fix: pin review harness model to available Vertex AI version - #188

Merged
gashcrumb merged 1 commit into
mainfrom
agent/187-fix-review-model
Sep 9, 2026
Merged

gashcrumb merged 1 commit into
mainfrom
agent/187-fix-review-model

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Summary

  • Pin the review harness model from the generic sonnet alias to claude-sonnet-4-5@20250929 in .fullsend/harness/review.yaml
  • The sonnet alias resolved to claude-sonnet-4-6, which is not provisioned on the repo's GCP Vertex AI endpoint, causing all review runs to fail since PR chore: upgrade fullsend review harness #181 merged
  • Using an explicit version pin avoids future breakage when the alias advances to unprovisioned models

Testing

  • Verified YAML syntax is valid
  • The fix can be validated by triggering /fs-review on any open PR and confirming the review agent starts without a model availability error

Closes #187

Post-script verification

  • Branch is not main/master (agent/187-fix-review-model)
  • Secret scan passed (gitleaks — 968ad8a9d2fcd3a530702bcf9a91446b9406577a..HEAD)
  • PR body secret scan passed (gitleaks — no-git)

The model: sonnet alias in the review harness config resolved to
claude-sonnet-4-6, which is not provisioned on the repo's GCP
Vertex AI endpoint. This caused all review runs to fail immediately
with zero tokens used since PR #181 merged.

Pin the model to claude-sonnet-4-5@20250929, which the error
message in run 34327800934 confirmed is available on the deployment.
Using an explicit version avoids future breakage if the sonnet
alias advances again before the deployment is updated.

Note: yarn lint:check and yarn prettier:check could not run
(yarn not available in sandbox). The change is a single-line YAML
config update that is not covered by ESLint or Prettier.

Closes #187

Assisted-by: claude-opus-4-6
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Triggers review agent dispatch label Sep 9, 2026
@sonarqubecloud

sonarqubecloud Bot commented Sep 9, 2026

Copy link
Copy Markdown

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure (validation failed after 1 iteration(s)) · Started 8:37 AM UTC · Completed 8:39 AM UTC

Commit: 1436b7a · View workflow run →

Runtime: claude · Model: sonnet → claude-sonnet-4-6 · Effort: high

@gashcrumb gashcrumb changed the title fix(#187): pin review harness model to available Vertex AI version fix: pin review harness model to available Vertex AI version Sep 9, 2026
@gashcrumb
gashcrumb merged commit cb1efdb into main Sep 9, 2026
44 of 48 checks passed
@gashcrumb
gashcrumb deleted the agent/187-fix-review-model branch September 9, 2026 10:15
@fullsend-ai-retro

fullsend-ai-retro Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 10:16 AM UTC · Completed 10:23 AM UTC

Commit: 1436b7a · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $2.56

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #188 — pin review harness model to available Vertex AI version

Timeline

  1. PR #181 merged (2026-09-09 08:11 UTC), introducing .fullsend/harness/review.yaml with model: sonnet to address Opus timeout issues on large PRs.
  2. The sonnet alias resolved to claude-sonnet-4-6, which is not provisioned on the repo's GCP Vertex AI endpoint — breaking all review agent runs.
  3. 08:25 — Retro agent detected the breakage and filed issue #187 with a clear description and proposed fix.
  4. 08:26–08:35 — Triage ($0.69) and code ($0.82) agents processed the issue autonomously, producing PR #188 with the correct 1-line fix in ~10 minutes.
  5. 08:35 — PR Semantic check failed: the code agent's title fix(#187): ... used # in the scope, which the repo's headerPattern regex rejects (run 34329926417).
  6. 08:37 — Review agent failed on PR fix: pin review harness model to available Vertex AI version #188 (run 34329926023) — chicken-and-egg: it used the same broken sonnet config this PR was fixing.
  7. 10:08 — Human renamed the PR title to fix: pin review harness model to available Vertex AI version.
  8. 10:09 — Human (gashcrumb) approved; 10:14 — merged.

What went well

  • Retro → triage → code pipeline worked excellently. The retro agent identified the breakage post-merge, filed a clear issue with a proposed fix, and the triage + code agents resolved it autonomously in ~10 minutes for $1.51 total. This is the detection-and-repair loop working as designed.
  • Code agent produced the correct fix on the first try — a clean 1-line model pin to claude-sonnet-4-5@20250929.
  • Low rework: only 1 iteration needed (modulo the PR title rename).

Existing issue coverage (no new proposals filed)

  • Model alias fallback/validation is extensively tracked in fullsend-ai/fullsend: #7026 (no automatic fallback when aliased model is not served), #6967 (fallback on policy_denied), #6964 (pre-dispatch model validation), #6883 (per-repo model catalog + preflight). This retro provides additional evidence: the sonnetclaude-sonnet-4-6 resolution failure blocked reviews on the entire repo until a manual pin was merged.
  • Config validation deadlock (review agent failing on the PR that fixes its own config) is covered by fullsend-ai/fullsend#6186. This retro provides a concrete instance: the review agent on PR fix: pin review harness model to available Vertex AI version #188 failed immediately with 0 tokens because it was using the broken config that the PR itself was fixing.

Proposal filed

1 proposal: document PR title scope character restrictions in AGENTS.md so the code agent avoids invalid scopes like (#187).

Proposals filed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-review Triggers review agent dispatch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Review harness model sonnet resolves to unavailable claude-sonnet-4-6 on Vertex AI

1 participant