Uh oh!
There was an error while loading. Please reload this page.
fix(provider): route non-native Cloudflare AI Gateway providers via the REST API - #44828
Merged
Merged
Conversation
…he REST API
The cloudflare-ai-gateway loader sent every non-OpenAI/Anthropic provider through
ai-gateway-provider's unified (compat) route, which posts to the gateway's universal
compat endpoint. That endpoint only fronts a handful of upstreams, so Google, xAI,
Alibaba, DeepSeek, Moonshot, etc. all failed with 'Bad Request: Invalid provider'.
Route those providers through Cloudflare's catalog-aware REST API instead
(/accounts/{id}/ai/v1/chat/completions), binding the request to the gateway with the
cf-aig-gateway-id header so it stays gateway-routed (analytics, caching, BYOK) rather
than bypassing it. models.dev ids (provider/model, dotted) pass through unchanged.
OpenAI/Anthropic native passthrough and Workers AI are unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes#44827
Type of change
What does this PR do?
Non-OpenAI/Anthropic models on the
cloudflare-ai-gatewayprovider all failed withBad Request: Invalid provider— Google, xAI, Alibaba, DeepSeek, Moonshot, every model.The loader routes anything that isn't
openai/,anthropic/, orworkers-ai/throughai-gateway-provider's unified (compat) route. That posts to the gateway's universal compat endpoint, which only fronts a handful of upstreams — the rest come back "Invalid provider".This routes those providers through Cloudflare's catalog-aware REST API (
/accounts/{id}/ai/v1/chat/completions) instead, and binds the request to the configured gateway with thecf-aig-gateway-idheader so it stays gateway-routed (analytics/caching/BYOK), not a bypass. OpenAI/Anthropic native passthrough and Workers AI are untouched. models.dev ids pass through unchanged.Pairs with the models.dev catalog rework that lists these providers: anomalyco/models.dev#4922.
How did you verify your code works?
Built the branch and ran every affected model through it against a live Cloudflare gateway (
opencode run --model cloudflare-ai-gateway/<provider>/<model>), base call plus each reasoning-effort variant — all 5 providers route and return content (Alibaba, DeepSeek, Moonshot, xAI fully green; Google works but Cloudflare intermittently returns "Failed to parse model output" on its gemini flash models regardless of routing — a non-deterministic upstream flake, not this change). Confirmed the requests appear in the gateway dashboard (so they're gateway-routed, not bypassing it).Checklist