Uh oh!
There was an error while loading. Please reload this page.
feat(server): register Provider::Cohere on Hub for chat-compat (#332) - #341
Merged
Conversation
Cohere exposes an OpenAI-compatible chat endpoint at `https://api.cohere.com/compatibility/v1/chat/completions` per <https://docs.cohere.com/reference/chat>. Before this commit `Provider::Cohere` was intentionally NOT Hub-registered (per #213 Phases 1-2 rerank-only plan), so a customer who selected Cohere in the dashboard and POSTed /v1/chat/completions got 503 ProviderUnavailable. The rerank surface at `/v1/rerank` continues to bypass the Bridge via aisix-proxy::rerank — this commit only adds the chat-compat dispatch. Fix 1. `OpenAiBridge` (`aisix-provider-openai/src/bridge.rs`) - New `COHERE_DEFAULT_BASE = "https://api.cohere.com/compatibility/v1"` constant for the `with_name("cohere")` variant's fallback base. - `default_base()` arm: `"cohere" => COHERE_DEFAULT_BASE`. - `normalize_api_base` arm + `normalize_canonical_cohere`: operators who paste the bare canonical host `https://api.cohere.com` (the rerank path / dashboard placeholder) get `/compatibility/v1` synthesized for chat. Non-canonical hosts pass through verbatim — operator's path on a custom host wins. 2. `build_hub()` (`aisix-server/src/main.rs`) - `hub.register(Provider::Cohere, Arc::new(OpenAiBridge::new().with_name("cohere")))` - Comment block updated: Jina alone stays rerank-only; Cohere now serves chat-compat via the bridge. `Provider::Cohere.default_base_url()` in `aisix-core` stays as `https://api.cohere.com` (bare host) because the rerank URL builder appends `/v1/rerank` to it. The bridge handles chat-compat in its own resolve_base(). Tests Three new bridge tests under `aisix-provider-openai/src/bridge.rs::tests`: - `cohere_default_base_targets_compatibility_v1` — empty `api_base` falls back to `/compatibility/v1`. - `cohere_api_base_tolerance_bare_host_synthesizes_compatibility_prefix` — bare host, trailing slash, full chat URL all normalize; a corporate-proxy host passes through unchanged. - `cohere_chat_compat_round_trips_openai_envelope` — end-to-end chat through `with_name("cohere")` returns the OpenAI envelope verbatim. Pins the contract Hub.register relies on. All 79 `aisix-provider-openai` tests pass; clippy clean; `aisix-server` builds clean. References (per CLAUDE.md §7) - Cohere chat-compat docs: https://docs.cohere.com/reference/chat - LiteLLM's Cohere chat-compat handler at https://github.com/BerriAI/litellm/blob/main/litellm/llms/cohere/chat/transformation.py uses the same `/compatibility/v1` namespace. - Portkey's Cohere routing: similar — points at `/compatibility/v1`. Closes#332
ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Free Run ID: 📒 Files selected for processing (3)
✅ Files skipped from review due to trivial changes (2)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughAdds Cohere support for OpenAI-compatible chat by normalizing Cohere canonical bases (synthesizing /compatibility/v1 when appropriate), registering Provider::Cohere with OpenAiBridge, and adding tests verifying base normalization and end-to-end chat compatibility. ChangesCohere OpenAI-Compatible Bridge Support
🎯 3 (Moderate) | ⏱️ ~20 minutes Note 🎁 Summarized by CodeRabbit FreeYour organization has reached its limit of developer seats under the Pro Plan. For new users, CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please add seats to your subscription by visiting https://app.coderabbit.ai/login.If you believe this is a mistake and have available seats, please assign one to the pull request author through the subscription management page using the link above. Comment |
There was a problem hiding this comment.
Pull request overview
This PR enables Cohere chat-completions dispatch by registering Provider::Cohere with the OpenAI-compatible bridge and teaching that bridge to target Cohere’s /compatibility/v1 namespace.
Changes:
- Adds Cohere default base URL and canonical host normalization in
OpenAiBridge. - Registers Cohere in the server Hub using
OpenAiBridge::with_name("cohere"). - Adds bridge tests covering Cohere default base resolution, normalization, and chat round-trip behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
crates/aisix-server/src/main.rs | Registers Cohere on the Hub and updates provider-registration comments. |
crates/aisix-provider-openai/src/bridge.rs | Adds Cohere base URL handling, normalization, and unit coverage for chat compatibility. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+774
to
+782
| /// `Provider::Cohere` is registered against the OpenAI-compatible | ||
| /// chat endpoint at `https://api.cohere.com/compatibility/v1` (per | ||
| /// <https://docs.cohere.com/reference/chat>). Cohere's rerank surface | ||
| /// at `/v1/rerank` continues to bypass the Bridge via | ||
| /// `aisix-proxy::rerank` — the bridge here only serves `chat/completions`, | ||
| /// `embeddings`, and the other OpenAI-shape endpoints the bridge | ||
| /// supports. The chat-compat namespace gives an exact OpenAI envelope | ||
| /// shape so `OpenAiBridge::with_name("cohere")` can serve it directly | ||
| /// (closes #332). |
…nt + build_hub registration tests Independent audit (CLAUDE.md §7) of PR #341 surfaced: - HIGH-1: `crates/aisix-core/src/models/model.rs:28-31` still documented `Provider::Cohere` as "exposed for /v1/rerank only" and "chat / generate APIs are not OpenAI-compatible". Both statements contradict the new Hub registration. Updated to describe the chat-compat path (https://api.cohere.com/compatibility/v1 per https://docs.cohere.com/reference/chat) and the model coverage caveat (command-r / command-a family on chat-compat; legacy command* not yet bridged). - MEDIUM-1: nothing test-side ties `Provider::Cohere` to `OpenAiBridge::with_name("cohere")` — the wiremock test in bridge.rs uses an override api_base so a regression where build_hub registered `OpenAiBridge::new()` (default name "openai") would still pass. Added two tests on `build_hub()`: - `build_hub_registers_cohere_chat_compat_variant` — asserts `hub.get(Provider::Cohere).unwrap().name() == "cohere"`. A `with_name("openai")` fallback would silently route Cohere chat to OpenAI's host via the bridge's `default_base()`. - `build_hub_does_not_register_jina_for_chat` — companion check so a future Jina chat-compat enablement is a deliberate change, not a silent regression. MEDIUM-2 (cohere streaming e2e), MEDIUM-3 (tool-use forwarding), MEDIUM-4 (model coverage gate) — scoped out of this PR per audit's recommendation, will land as separate enhancements after the chat-compat baseline merges. LOW findings deferred (passthrough comment staleness; endpoint- suffix corner case).
Comment on lines
+795
to
+798
| hub.register( | ||
| Provider::Cohere, | ||
| Arc::new(OpenAiBridge::new().with_name("cohere")), | ||
| ); |
Uh oh!
There was an error while loading. Please reload this page.
janiussyafiq added a commit
that referenced
this pull request
May 20, 2026
Integrate origin/main (commit 2c1d485 = post-PR-#326 / #348 plus #330 / #341 / #343 / #345 / #346) into this branch via `git merge --squash` to clear PR #344's lingering `mergeable: dirty` state. Conflict on `docs/quickstart/self-hosted.md` was a 3-way-merge-base artifact: base (3596c0a) read `- a reachable etcd instance`, main changed `a` → `A` (via #326), this branch additionally inserted the glossary link. Both changes are wanted; resolution per Umar's approved plan was `git checkout --ours`, which preserves the branch's self-hosted.md state (already integrates capital A + glossary link + first-time-build paragraph + keep-running framing). Other 4 overlapping doc files auto-merged cleanly (`bootstrap-config.md`, `core-concepts.md`, `first-model-first-key-first-request.md`, `openai-sdk.md`). Code files all auto-merged cleanly. Additional Copilot review (post-`167196a` cycle) addressed: - `docs/index.md:7` — change link display text from `[data-plane]` to `[data plane]` to match the canonical glossary term. The URL anchor `#data-plane` stays kebab-case (matches the glossary heading's auto-anchor); only the display text changes. Comment id 3271145422. - `docs/quickstart/openai-sdk.md:43` — change `All three steps below` to `All commands below`. The Install-the-SDK section has two command blocks (mkdir+cd, npm install), not three; the prior wording originated from a mental model (mkdir, cd, install) that doesn't match the typographic count of code blocks under the heading. Comment id 3271145458. Copilot's third comment on `docs/overview/core-concepts.md` Observability Exporter wording (id 3271145444) auto-resolves via this merge — main's #326 rewrite supersedes the branch's pre-#326 wording at that location ("ships per-request span telemetry… OTLP/HTTP-compatible backend…" replaces "Use this concept when documenting…"). No separate edit needed; the merge IS the fix.
janiussyafiq added a commit
that referenced
this pull request
May 20, 2026
…ickstart-polish Resolve PR #344's lingering mergeable: dirty state by linking the branch history to origin/main (2c1d485 = post-#326 / #348 / #330 / #341 / #343 / #345 / #346). The squash-merge commit landed earlier (e2af197) integrated main's content into the branch tree but did not link the histories, so GitHub's mergeable computation still saw the 3-way-merge-base artifact conflict on docs/quickstart/self-hosted.md (a vs A + the glossary link / "In another terminal" vs "Keep the gateway running" framing). This explicit merge commit ties the branch to main's history. Self-hosted.md conflict resolved by taking OUR side — the branch's edits already contain main's substantive changes (capital A, first-time-build paragraph) plus this PR's additions (glossary link, keep-running framing, YOUR_ADMIN_KEY note, config.yaml location anchor). The auto-merge of first-model-first-key-first-request.md duplicated the :::warning callout that was already integrated via the squash commit; removed the duplicate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cohere exposes an OpenAI-compatible chat endpoint at
https://api.cohere.com/compatibility/v1/chat/completions. Until this PRProvider::Coherewas intentionally NOT Hub-registered (per #213 Phases 1-2 rerank-only plan), so customers who selected Cohere in the dashboard and POSTed/v1/chat/completionsgot 503 ProviderUnavailable. The rerank surface at/v1/rerankcontinues to bypass the Bridge viaaisix-proxy::rerank— this PR only adds the chat-compat dispatch.Surfaced by AISIX-Cloud's source-blind E2E matrix audit on PR #349 (D3.1 OpenAI-adapter long-tail), which held back Cohere chat-compat scenarios pending this Hub registration.
Fix
OpenAiBridge—aisix-provider-openai/src/bridge.rsCOHERE_DEFAULT_BASE = "https://api.cohere.com/compatibility/v1"constant for thewith_name("cohere")variant's fallback base.default_base()arm:"cohere" => COHERE_DEFAULT_BASE.normalize_canonical_cohere: operators who paste the bare canonical hosthttps://api.cohere.com(the rerank path / dashboard placeholder) get/compatibility/v1synthesized for chat. Non-canonical hosts pass through verbatim.build_hub—aisix-server/src/main.rshub.register(Provider::Cohere, Arc::new(OpenAiBridge::new().with_name("cohere"))).NOT touched
Provider::Cohere.default_base_url()inaisix-corestays ashttps://api.cohere.com(bare host) — the rerank URL builder appends/v1/rerank. The bridge handles chat-compat in its ownresolve_base().Tests
Three new bridge tests:
cohere_default_base_targets_compatibility_v1api_base→ bridge falls back to/compatibility/v1cohere_api_base_tolerance_bare_host_synthesizes_compatibility_prefixcohere_chat_compat_round_trips_openai_envelopewith_name("cohere")returns the OpenAI envelope verbatimAll 79
aisix-provider-openaitests pass; clippy clean;aisix-serverbuilds clean.References (per CLAUDE.md §7)
/compatibility/v1/chat/completionsshape/compatibility/v1namespace/compatibility/v1tooTest plan
defaultBase: 'https://api.cohere.com/compatibility/v1'(separate AISIX-Cloud PR; bridge tolerance covers bare-host PKs migration-free)Closes#332
Summary by CodeRabbit
New Features
Documentation
Tests