Skip to content

Spend the review's own budget waiting out a transient provider limit - #162

Closed
morgaesis wants to merge 1 commit into
mainfrom
fix/transient-retry-uses-budget
Closed

Spend the review's own budget waiting out a transient provider limit#162
morgaesis wants to merge 1 commit into
mainfrom
fix/transient-retry-uses-budget

Conversation

@morgaesis

Copy link
Copy Markdown
Contributor

A retryable provider status got three attempts inside about six seconds and then failed the review, however much time the review actually held. On 2026-08-20 three pull requests failed with Model provider unavailable after HTTP 429 three times in 5.2 seconds, while the same log line recorded budget_remaining=326.0s. The upstream limit cleared minutes later, and the provider had said as much in the response body: z-ai/glm-5.2 is temporarily rate-limited upstream. Please retry shortly. The reviewer was funded to wait and did not.

Retries past the fixed allowance are now decided by the phase's remaining budget rather than by a count. They are capped at twelve so a provider that rejects instantly cannot spin a phase away, and each one must leave enough time for the attempt it funds, so a wait that reaches the deadline and never issues its request cannot happen. Backoff doubles past the fixed allowance and is bounded at 20 seconds; a provider-supplied Retry-After still wins. An unbudgeted caller keeps exactly the previous allowance, so local and bring-your-own-key runs are unchanged.

This narrows the failure rather than removing it, and the remaining half is worth stating. Hosted requests pin one upstream provider with allow_fallbacks: false, so that provider's rate limit is a total outage. Two probes seconds apart during the incident: the pinned request returned 429, and the identical request with the pin removed and zero-retention still required returned 200, served by another endpoint. Capacity existed throughout and the pin forbade reaching it. Widening the pin to an ordered set changes the signed profile schema and the response-identity validator, so it is proposed separately rather than folded in here.

A retryable provider status got three attempts inside about six seconds and
then failed the review, however much time the review actually held. On
2026-08-20 three pull requests failed with `Model provider unavailable` after
HTTP 429 three times in 5.2 seconds while the log recorded
`budget_remaining=326.0s`. The upstream limit cleared minutes later, and the
provider had said so: `temporarily rate-limited upstream. Please retry
shortly`. The reviewer was funded to wait and did not.
Retries past the fixed allowance are now decided by the phase's remaining
budget rather than by a count, capped at twelve and required to leave enough
time for the attempt they fund, so a wait that reaches the deadline and never
issues its request cannot happen. Backoff doubles past the fixed allowance and
is bounded at 20 seconds, and a provider-supplied `Retry-After` still wins.
An unbudgeted caller keeps exactly the previous allowance, so local and
bring-your-own-key runs are unchanged.
This narrows the failure rather than removing it. The deeper cause is that
hosted requests pin one upstream provider with fallbacks disabled, so one
provider's rate limit is an outage while the same request without the pin is
served immediately by another endpoint.
@morgaesis

Copy link
Copy Markdown
ContributorAuthor

Superseded by PR #171, which keeps extended retries phase-scoped to budgeted Review calls and preserves three attempts for attribution and every other non-review phase. The replacement adds deadline-reserve, provider-hint, unbudgeted, and admission regressions on current main.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@morgaesis