Retry model-catalog 429s and stop forwarding unresolved Copilot model aliases by Copilot · Pull Request #53456 · github/gh-aw

Copilot · 2026-08-17T18:28:19Z

A transient HTTP 429 from the AWF model-catalog endpoint caused the Copilot harness to treat the catalog as empty, skip alias resolution, and invoke Copilot with the raw alias (e.g. sol) instead of the concrete model. The API proxy then rejected the request with a misleading "no AI credits pricing" error, even though the alias and pricing were valid — the real cause was a transient catalog fetch failure.

Retry transient 429s in the model-catalog fetch

fetchModelsFromUrl() now treats HTTP 429 as retryable alongside 503, bounded by AWF_MODELS_URL_MAX_ATTEMPTS.
Honors a valid Retry-After response header, capped at AWF_MODELS_URL_RETRY_MAX_MS, falling back to the existing exponential backoff otherwise.
Retry/exhaustion log messages report the actual status code instead of hard-coding 503.
Permanent 4xx responses (400/401/403) remain fail-fast, unretried.

Stop invoking Copilot with a known, unresolved alias

resolveConfiguredCopilotModel() now throws a dedicated ModelAliasResolutionError when the configured model is a known alias but the catalog built from awf-reflect data is empty, instead of silently returning the alias unchanged.
Concrete (non-alias) models continue to resolve/normalize without requiring a catalog.

// Before: empty catalog -> alias forwarded unchangedresolveConfiguredCopilotModel({configuredModel: "sol", aliasMap,reflectData: null});// => "sol" (forwarded to Copilot as-is, proxy rejects with misleading pricing error)// After: empty catalog -> throws, caller must handleresolveConfiguredCopilotModel({configuredModel: "sol", aliasMap,reflectData: null});// => throws ModelAliasResolutionError

Bounded catalog refresh before failing

applyCopilotModelAliasResolution() in the Copilot harness is now async. On ModelAliasResolutionError, it performs one bounded awf-reflect refetch and retries resolution once.
If the refresh still can't produce a catalog, it logs an explicit catalog/alias diagnostic and exits before Copilot is spawned — the unresolved alias never reaches the API proxy.

Tests

New cases in awf_reflect.test.cjs cover 429 retry-to-success, retry exhaustion, Retry-After capping, and non-retry of 400/401/403.
New cases in resolve_model_alias.test.cjs and copilot_harness.test.cjs cover the alias-vs-empty-catalog error, the refresh-then-succeed path, and the refresh-then-fail path that aborts before spawning Copilot.

Note: the plan also called for a smoke fixture simulating a 429-then-200 models endpoint; no existing smoke/e2e fixture harness for the AWF api-proxy was found in this repo (the api-proxy itself lives outside gh-aw), so that item was left out of this change.

FixesModel catalog 429 leaves alias unresolved and produces misleading pricing failure #52782

Run: https://github.com/github/gh-aw/actions/runs/32066505163> Generated by 👨‍🍳 PR Sous Chef · gpt54 · 5.87 AIC · ⌖ 6.48 AIC · ⊞ 8.4K · ◷

_{Comment /souschef to run again}

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

github-actions · 2026-08-17T18:50:15Z

✅ Ponytail Reviewer completed successfully!

Generated by Ponytail Reviewer for #53456

github-actions · 2026-08-17T18:50:17Z

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

github-actions · 2026-08-17T18:50:26Z

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

github-actions · 2026-08-17T18:50:32Z

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch test file diff

🧪 Test quality analysis by Test Quality Sentinel

github-actions · 2026-08-17T18:50:33Z

✅ Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR #53456 does not have the 'implementation' label and has 0 new lines of code in business logic directories (threshold: 100).

🏗️ ADR gate enforced by Design Decision Gate 🏗️

github-actions · 2026-08-17T18:54:26Z

Comment Memory

reviewed_at: 2026-08-17T00:00:00Z
review_event: COMMENT
top_themes:
- transient 429 handling for model catalog
- alias-resolution fail-fast behavior
- test coverage added for retry and alias-refresh paths
files_reviewed:
- actions/setup/js/awf_reflect.cjs
- actions/setup/js/awf_reflect.test.cjs
- actions/setup/js/copilot_harness.cjs
- actions/setup/js/copilot_harness.test.cjs
- actions/setup/js/resolve_model_alias.cjs
- actions/setup/js/resolve_model_alias.test.cjs
comment_count: 0

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · gpt54 · 9.08 AIC · ⌖ 6.18 AIC · ⊞ 6.9K · ◷
_{Comment /review to run again}

github-actions

Verdict

I don't see a blocking defect in the changed lines.

Notes

I checked the new 429 retry path, the Retry-After plumbing into withRetry, the alias-resolution fail-fast behavior, and the new tests around bounded refresh / abort-before-spawn. The control flow is consistent with the stated goal: transient catalog failures now get one extra recovery path, and known aliases are no longer silently forwarded into a misleading downstream error.

🔎 Code quality review by PR Code Quality Reviewer · gpt54 · 9.08 AIC · ⌖ 6.18 AIC · ⊞ 6.9K
_{Comment /review to run again}

github-actions

Ponytail review (over-engineering only).

L395 (copilot_harness.cjs): dead return after process.exit(1) — see inline comment.

Everything else (retry-status Set, Retry-After extraction, dedicated ModelAliasResolutionError, single bounded refresh-and-retry) is proportionate to the bug being fixed; no speculative abstractions or unused flexibility found.

net: -1 lines possible.

Generated by ✂️ Ponytail Reviewer for #53456 · auto · 26.3 AIC · ⌖ 3.59 AIC · ⊞ 7.2K
_{Comment /ponytail to run again}

github-actions · 2026-08-17T18:56:02Z

+ }
+ logger(`copilot model alias resolution failed: model-catalog retrieval prevented alias resolution for '${configuredModel}' after a bounded refresh — refusing to start Copilot with an unresolved alias`);
+ process.exit(1);
+ return configuredModel; // unreachable, keeps TypeScript control-flow analysis happy


L395: delete: return configuredModel; after process.exit(1), justified only by a comment about "TypeScript control-flow analysis". Nothing replaces it — the function isn't typed as non-nullable-returning and no lint requires this line.

Copilot

Pull request overview

Adds model-catalog throttling recovery and prevents unresolved Copilot aliases from reaching the API proxy.

Changes:

Retries HTTP 429 responses with bounded backoff and Retry-After support.
Refreshes the catalog once before aborting unresolved alias resolution.
Adds focused retry and alias-resolution tests.

Show a summary per file

File	Description
`actions/setup/js/awf_reflect.cjs`	Adds retry handling for catalog 429 responses.
`actions/setup/js/awf_reflect.test.cjs`	Tests retry, exhaustion, delay, and fail-fast behavior.
`actions/setup/js/resolve_model_alias.cjs`	Introduces explicit unresolved-alias failure handling.
`actions/setup/js/resolve_model_alias.test.cjs`	Tests alias failure and concrete-model behavior.
`actions/setup/js/copilot_harness.cjs`	Refreshes the catalog once before terminating.
`actions/setup/js/copilot_harness.test.cjs`	Tests refresh success and abort behavior.

Review details

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Files reviewed: 6/6 changed files
Comments generated: 2
Review effort level: Balanced

 const catalog = buildCatalogFromReflect(options.reflectData);
 if (catalog.length === 0) {
- logger(`copilot model alias resolution skipped (empty catalog from awf-reflect)`);
- return configuredModel;
+ // Do not silently forward a known alias unchanged: the API proxy rejects unresolved
+ // aliases with a misleading "no AI credits pricing" error instead of surfacing the
+ // real cause (catalog retrieval failure). Callers should retry/refresh the catalog
+ // and, if it still cannot be built, stop before invoking Copilot.
+ logger(`copilot model alias resolution: catalog unavailable from awf-reflect for known alias '${configuredModel}'`);
+ throw new ModelAliasResolutionError(configuredModel);


+function extractRetryAfterHeader(res) {
+ try {
+ const retryAfter = res?.headers?.get?.("retry-after");
+ return retryAfter != null ? { "retry-after": retryAfter } : undefined;
+ } catch {
+ return undefined;
+ }
+}


github-actions

The changes are well-structured and correctly address the 429 retry gap described in #52782.

What the PR does:

Adds 429 to AWF_MODELS_URL_RETRYABLE_STATUSES alongside 503, with proper Retry-After header forwarding to the existing withRetry backoff logic.
Replaces the silent alias passthrough on empty catalog with ModelAliasResolutionError, a safer contract: callers now know to retry/refresh rather than forwarding an unresolved alias.
Adds a bounded single-refresh attempt in applyCopilotModelAliasResolution, followed by a hard process.exit(1) guard if the catalog is still unavailable — preventing the misleading "no AI credits pricing" proxy error.

Quality: comprehensive unit tests cover the 429 retry path, Retry-After capping, permanent-4xx no-retry, alias-resolution error on empty catalog, successful refresh, and exhausted-refresh exit guard. No issues found.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · sonnet46 · 31.7 AIC · ⌖ 7.1 AIC · ⊞ 5.6K

github-actions

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd — commenting with improvement suggestions; not blocking.

📋 Key Themes & Highlights

Key Themes

Log message accuracy — Two log messages fire before their described action happens, and one conflates distinct failure modes (null catalog vs. empty-models catalog). Inline comments #1, #3, and #4 cover these.
Retry-After cap test specificity — The timer-based test for the 2000 ms cap is structurally correct but its description doesn't read as a specification of the capping behaviour. Inline comment #2 covers this.
Missing-refetch path clarity — When refetchReflectData is not supplied (e.g. in tests), the fallback is silent null but the exhaustion log says "after a bounded refresh" regardless. Inline comment #1 covers this.

Positive Highlights

✅ Clean extraction of ModelAliasResolutionError as a dedicated typed error — callers can now distinguish catalog failure from other resolution errors without string-matching.
✅ Retry-After cap wired correctly through getRetryAfterMs → maxDelayMs; no separate cap logic needed.
✅ Exhaustive parametric tests for 400/401/403 non-retry cases — good spec coverage of permanent-failure fast-path.
✅ Test cleanup guard for awf-config.json is well-structured; restores pre-existing content rather than blindly deleting.
✅ extractRetryAfterHeader is defensively coded (try/catch, optional chaining) and correctly narrowed to the retry-after key only — avoids accidental data leakage from headers.
✅ AWF_MODELS_URL_RETRYABLE_STATUSES as a Set is a clean extension point for future retryable codes.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 · 88.5 AIC · ⌖ 9.09 AIC · ⊞ 7.7K
_{Comment /matt to run again}

github-actions · 2026-08-17T19:01:18Z

+ if (!(retryErr instanceof ModelAliasResolutionError)) {
+ throw retryErr;
+ }
+ logger(`copilot model alias resolution failed: model-catalog retrieval prevented alias resolution for '${configuredModel}' after a bounded refresh — refusing to start Copilot with an unresolved alias`);


[/diagnosing-bugs] Misleading diagnostic when refetchReflectData is not provided: no refetch is attempted, but the log says "after a bounded refresh", which obscures debugging.
💡 Suggested fix
constdidRefetch=!!options.refetchReflectData;logger(`copilot model alias resolution failed: model-catalog retrieval prevented alias resolution for '${configuredModel}' ${didRefetch ? "after a bounded refresh" : "(no refetch available)"} — refusing to start Copilot with an unresolved alias`);
Keeps the message accurate whether or not a refetch was possible.
@copilot please address this.

github-actions · 2026-08-17T19:01:18Z

+ expect(fetchMock).toHaveBeenCalledTimes(1);
+
+ await vi.advanceTimersByTimeAsync(1);
+ const result = await run;


[/tdd] The Retry-After test verifies the second fetch fires after AWF_MODELS_URL_RETRY_MAX_MS but does not assert any delay is actually observed for the cap boundary — only the call count is checked. The cap behaviour (capped from 3000 ms → 2000 ms) is implicit.
💡 Suggested addition
Add a timing assertion to make the spec self-documenting:
// After advancing exactly RETRY_MAX_MS-1, retry must not have fired yetawaitvi.advanceTimersByTimeAsync(AWF_MODELS_URL_RETRY_MAX_MS-1);expect(fetchMock).toHaveBeenCalledTimes(1);// already present// Now one more ms triggers it — confirms the cap is 2000ms, not the raw 3000ms headerawaitvi.advanceTimersByTimeAsync(1);// existing assertions follow
The second boundary check is present; consider naming the test description "...caps Retry-After to AWF_MODELS_URL_RETRY_MAX_MS" to make it a specification.
@copilot please address this.

github-actions · 2026-08-17T19:01:18Z

+ if (!(err instanceof ModelAliasResolutionError)) {
+ throw err;
+ }
+ logger(`copilot model alias resolution: retrying awf-reflect model-catalog fetch once before failing for '${configuredModel}'`);


[/diagnosing-bugs] Log message fires before the refetch is attempted, so if the refetch call itself throws or hangs, the log misleadingly suggests the retry already started. Move the log inside the successful-refetch branch, or rephrase to present tense.
💡 Suggested rephrase
logger(`copilot model alias resolution: initial catalog empty for '${configuredModel}'; attempting one awf-reflect refresh`);
Then after the refetch:
logger(`copilot model alias resolution: refresh complete, retrying alias resolution for '${configuredModel}'`);
This makes the log sequence reflect what actually happened, not what is about to happen.
@copilot please address this.

github-actions · 2026-08-17T19:01:18Z

+ // real cause (catalog retrieval failure). Callers should retry/refresh the catalog
+ // and, if it still cannot be built, stop before invoking Copilot.
+ logger(`copilot model alias resolution: catalog unavailable from awf-reflect for known alias '${configuredModel}'`);
+ throw new ModelAliasResolutionError(configuredModel);


[/diagnosing-bugs]ModelAliasResolutionError is thrown even when reflectData is non-null but every endpoint has an empty models array — not just when the catalog fetch failed. The log message says "catalog unavailable from awf-reflect" which is accurate for the null case, but misleading when data was returned but had no models. Consider splitting the log message:
💡 Suggested clarification
constreason=options.reflectData==null ? "catalog fetch returned no data" : "all awf-reflect endpoints had empty model lists";logger(`copilot model alias resolution: ${reason} for known alias '${configuredModel}'`);thrownewModelAliasResolutionError(configuredModel);
This helps operators distinguish a transient 429 (null reflectData) from a misconfigured endpoint (empty models).
@copilot please address this.

github-actions · 2026-08-17T19:01:18Z

+ const refreshedReflectData = { endpoints: [{ configured: true, provider: "copilot", models: ["gpt-5.6-sol"] }] };
+ const refetch = vi.fn().mockResolvedValue(refreshedReflectData);
+
+ const resolved = await applyCopilotModelAliasResolution({


[/tdd] The "leaves a concrete configured model unchanged" test passes awfReflectData: null but the concrete model copilot/gpt-5.6-sol also appears as a value in the alias map. The test exercises the "not an alias key" branch — but does not explicitly assert that resolveConfiguredCopilotModel was not called a second time via refetch. The intent is clear but the assertion on refetch only says .not.toHaveBeenCalled(), which is sufficient; just worth noting the test name could be "does not refetch when configured model is concrete (not an alias key)" to read as a spec.
@copilot please address this.

pelikhan · 2026-08-17T20:14:28Z

@copilot run pr-finisher skill

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-08-17T20:39:38Z

@copilot run pr-finisher skill

Completed a pr-finisher pass and pushed f50d008. Addressed two actionable correctness gaps (known-alias unresolved fallback and HTTP-date Retry-After parsing), validated locally with fmt/lint/test-unit/test/recompile, and scanned changed files for secrets.

gh-aw-bot · 2026-08-17T20:41:07Z

@copilot this PR looks close.

Please review and address these unresolved threads, then run the pr-finisher skill and hand the PR back to maintainers with a short validation summary.

Unresolved review threads (newest first):

github-actions: Retry model-catalog 429s and stop forwarding unresolved Copilot model aliases #53456 (comment)
github-actions: Retry model-catalog 429s and stop forwarding unresolved Copilot model aliases #53456 (comment)
github-actions: Retry model-catalog 429s and stop forwarding unresolved Copilot model aliases #53456 (comment)
github-actions: Retry model-catalog 429s and stop forwarding unresolved Copilot model aliases #53456 (comment)
github-actions: Retry model-catalog 429s and stop forwarding unresolved Copilot model aliases #53456 (comment)
copilot-pull-request-reviewer: Retry model-catalog 429s and stop forwarding unresolved Copilot model aliases #53456 (comment)
copilot-pull-request-reviewer: Retry model-catalog 429s and stop forwarding unresolved Copilot model aliases #53456 (comment)
github-actions: Retry model-catalog 429s and stop forwarding unresolved Copilot model aliases #53456 (comment)

Please also refresh the branch if GitHub allows it.

Run: https://github.com/github/gh-aw/actions/runs/32066505163

Generated by 👨🍳 PR Sous Chef

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 5.87 AIC · ⌖ 6.48 AIC · ⊞ 8.4K · ◷
_{Comment /souschef to run again}

github-actions · 2026-08-18T22:22:32Z

🎉 This pull request is included in a new release.

Release: v0.87.1

Initial plan
675344d

CopilotAI assigned Copilot and pelikhan Aug 17, 2026

Copilotstarted work on behalf of pelikhan August 17, 2026 18:28 View session

CopilotAI linked an issue Aug 17, 2026 that may be closed by this pull request
Model catalog 429 leaves alias unresolved and produces misleading pricing failure #52782
Closed

Retry model-catalog 429 and refuse to forward unresolved Copilot aliases
fb18ae0
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

CopilotAI changed the title ~~[WIP] Fix model catalog 429 issue causing unresolved alias~~Retry model-catalog 429s and stop forwarding unresolved Copilot model aliasesAug 17, 2026

Copilotfinished work on behalf of pelikhan August 17, 2026 18:47

CopilotAI requested a review from pelikhan August 17, 2026 18:47

pelikhan marked this pull request as ready for review August 17, 2026 18:49

CopilotAI balanced review requested due to automatic review settings August 17, 2026 18:49

Copilotstarted reviewing on behalf of pelikhan August 17, 2026 18:49 View session

github-actionsBot reviewed Aug 17, 2026
View reviewed changes

github-actionsBot mentioned this pull request Aug 17, 2026
[aw] No-Op Runs #48838
Open

github-actionsBot reviewed Aug 17, 2026
View reviewed changes

CopilotAI reviewed Aug 17, 2026
View reviewed changes

github-actionsBot approved these changes Aug 17, 2026
View reviewed changes

github-actionsBot reviewed Aug 17, 2026
View reviewed changes

github-actionsBot mentioned this pull request Aug 17, 2026
[pr-sous-chef] PR Sous Chef run report #53451
Closed

Merge branch 'main' into copilot/fix-model-catalog-429-issue
5d4dbb1

Copilotstarted work on behalf of pelikhan August 17, 2026 20:14 View session

CopilotAIand others added 2 commits August 17, 2026 20:18

chore: start pr-finisher pass
23a87c9
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Fix alias-resolution fallback and HTTP-date Retry-After parsing
f50d008
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilotfinished work on behalf of pelikhan August 17, 2026 20:40

Merge branch 'main' into copilot/fix-model-catalog-429-issue
b3ec576

Copilotstarted work on behalf of gh-aw-bot August 17, 2026 20:41 View session

Merge branch 'main' into copilot/fix-model-catalog-429-issue
9434b13

pelikhan merged commit d6f8189 into mainAug 17, 2026
9 checks passed

pelikhan deleted the copilot/fix-model-catalog-429-issue branch August 17, 2026 20:51

Copilot stopped work on behalf of gh-aw-bot due to an error August 17, 2026 20:52
The session was cancelled by the user.

CopilotAI requested a review from gh-aw-bot August 17, 2026 20:52

Uh oh!

Conversation

CopilotAI commented Aug 17, 2026• edited by github-actions BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Retry transient 429s in the model-catalog fetch

Stop invoking Copilot with a known, unresolved alias

Bounded catalog refresh before failing

Tests

Uh oh!

github-actionsBot commented Aug 17, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot commented Aug 17, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot commented Aug 17, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot commented Aug 17, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot commented Aug 17, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot commented Aug 17, 2026

Comment Memory

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Verdict

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBotAug 17, 2026

Choose a reason for hiding this comment

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Skills-Based Review 🧠

Key Themes

Positive Highlights

Uh oh!

github-actionsBotAug 17, 2026

Choose a reason for hiding this comment

Uh oh!

github-actionsBotAug 17, 2026

Choose a reason for hiding this comment

Uh oh!

github-actionsBotAug 17, 2026

Choose a reason for hiding this comment

Uh oh!

github-actionsBotAug 17, 2026

Choose a reason for hiding this comment

Uh oh!

github-actionsBotAug 17, 2026

Choose a reason for hiding this comment

Uh oh!

pelikhan commented Aug 17, 2026

Uh oh!

CopilotAI commented Aug 17, 2026

Uh oh!

gh-aw-bot commented Aug 17, 2026

Uh oh!

Uh oh!

github-actionsBot commented Aug 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

CopilotAI commented Aug 17, 2026•
edited by github-actions Bot
Loading

github-actionsBot commented Aug 17, 2026•
edited
Loading

github-actionsBot commented Aug 17, 2026•
edited
Loading

github-actionsBot commented Aug 17, 2026•
edited
Loading

github-actionsBot commented Aug 17, 2026•
edited
Loading

github-actionsBot commented Aug 17, 2026•
edited
Loading