Uh oh!
There was an error while loading. Please reload this page.
Spend the review's own budget waiting out a transient provider limit - #162
Closed
morgaesis wants to merge 1 commit into
Closed
Spend the review's own budget waiting out a transient provider limit#162morgaesis wants to merge 1 commit into
morgaesis wants to merge 1 commit into
Conversation
A retryable provider status got three attempts inside about six seconds and then failed the review, however much time the review actually held. On 2026-08-20 three pull requests failed with `Model provider unavailable` after HTTP 429 three times in 5.2 seconds while the log recorded `budget_remaining=326.0s`. The upstream limit cleared minutes later, and the provider had said so: `temporarily rate-limited upstream. Please retry shortly`. The reviewer was funded to wait and did not. Retries past the fixed allowance are now decided by the phase's remaining budget rather than by a count, capped at twelve and required to leave enough time for the attempt they fund, so a wait that reaches the deadline and never issues its request cannot happen. Backoff doubles past the fixed allowance and is bounded at 20 seconds, and a provider-supplied `Retry-After` still wins. An unbudgeted caller keeps exactly the previous allowance, so local and bring-your-own-key runs are unchanged. This narrows the failure rather than removing it. The deeper cause is that hosted requests pin one upstream provider with fallbacks disabled, so one provider's rate limit is an outage while the same request without the pin is served immediately by another endpoint.
morgaesis
commented
Aug 25, 2026
ContributorAuthor
Superseded by PR #171, which keeps extended retries phase-scoped to budgeted |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A retryable provider status got three attempts inside about six seconds and then failed the review, however much time the review actually held. On 2026-08-20 three pull requests failed with
Model provider unavailableafter HTTP 429 three times in 5.2 seconds, while the same log line recordedbudget_remaining=326.0s. The upstream limit cleared minutes later, and the provider had said as much in the response body:z-ai/glm-5.2 is temporarily rate-limited upstream. Please retry shortly. The reviewer was funded to wait and did not.Retries past the fixed allowance are now decided by the phase's remaining budget rather than by a count. They are capped at twelve so a provider that rejects instantly cannot spin a phase away, and each one must leave enough time for the attempt it funds, so a wait that reaches the deadline and never issues its request cannot happen. Backoff doubles past the fixed allowance and is bounded at 20 seconds; a provider-supplied
Retry-Afterstill wins. An unbudgeted caller keeps exactly the previous allowance, so local and bring-your-own-key runs are unchanged.This narrows the failure rather than removing it, and the remaining half is worth stating. Hosted requests pin one upstream provider with
allow_fallbacks: false, so that provider's rate limit is a total outage. Two probes seconds apart during the incident: the pinned request returned 429, and the identical request with the pin removed and zero-retention still required returned 200, served by another endpoint. Capacity existed throughout and the pin forbade reaching it. Widening the pin to an ordered set changes the signed profile schema and the response-identity validator, so it is proposed separately rather than folded in here.