Uh oh!
There was an error while loading. Please reload this page.
fix(openai): normalise OpenRouter non-stream message.reasoning → reasoning_content (#648) - #501
Conversation
…oning_content (#648) OpenRouter (and some OpenAI-compatible aggregators) return a reasoning model's chain-of-thought at `message.reasoning` on the non-stream path, NOT the DeepSeek-canonical `message.reasoning_content`. `OpenAiResponseMessage` was a closed struct that only deserialized `reasoning_content`, so `message.reasoning` was dropped at decode. A reasoning model (e.g. z-ai/glm-4.6 via OpenRouter) that emits its whole answer as reasoning then reached the customer with BOTH `content` AND `reasoning_content` empty — observed on the real-chain e2e (status 200, completion_tokens=230, empty customer fields). The streaming path already handles this via the per-key `reasoning_field` override, but that lift is streaming/`delta`-only (extract_reasoning_field hard-rejects non-`delta` paths), so non-stream had no path to surface it. Fix (auto-normalize, no per-key config needed): - Capture `message.reasoning` on OpenAiResponseMessage (was dropped). - In response_into_chat_response, lift it into the canonical `reasoning_content` extra slot when canonical reasoning_content is absent/empty. The DeepSeek-canonical field takes precedence when both are present; empty `reasoning` adds no noise field (same skip-empty rule as #466). Symmetric with the existing non-stream reasoning_content capture. Non-OpenRouter upstreams omit `reasoning` (serde default) → unaffected. Tests: 3 new unit tests (OpenRouter reasoning→canonical; canonical-wins precedence; empty-reasoning adds no field) + the existing 8 reasoning tests pass. clippy + fmt clean; sibling crates (azure-openai, proxy) build.
Warning Review limit reached
More reviews will be available in 5 minutes and 21 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Free Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThis PR extends the OpenAI provider's wire format to handle OpenRouter-style reasoning. The struct gains an optional ChangesOpenRouter Reasoning Normalization
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Note 🎁 Summarized by CodeRabbit FreeYour organization has reached its limit of developer seats under the Pro Plan. For new users, CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please add seats to your subscription by visiting https://app.coderabbit.ai/login.If you believe this is a mistake and have available seats, please assign one to the pull request author through the subscription management page using the link above. Comment |
…normalisation
The empty-canonical-falls-through case is the one branch exercising the
real precedence logic (`reasoning_content.filter(!empty).or(reasoning)`):
an empty `reasoning_content` ("") alongside a real OpenRouter `reasoning`
must surface `reasoning`. This test FAILS against the pre-fix
canonical-only code, so it's a genuine regression guard (#648).moonming
commented
Jun 3, 2026
Audit remediation (REQUEST-CHANGES → all 3 MEDIUM addressed)MEDIUM-1 — non-discriminating tests. Added a 4th test MEDIUM-2 — scope read as OpenRouter-only. Added a "Scope / blast radius" section to the PR body: MEDIUM-3 — streaming-parity not tracked. Filed #502 (auto-normalize Local gate: |
Uh oh!
There was an error while loading. Please reload this page.
…ponse (#684) gpt-4o-audio chat models return generated audio at message.audio when the request asks for the audio output modality. The request-side modalities/audio params already forward via the OpenAI bridge's flatten extra, so the upstream generates audio — but OpenAiResponseMessage didn't model audio, so it was dropped at deserialize (customer got 200 + tokens, no audio). Adds an audio field + surfaces it via ChatMessage.extra (same mechanism #501 used for reasoning). Null/absent audio adds no field; non-audio upstreams unaffected. Fixesapi7/AISIX-Cloud#684. Streaming-audio parity tracked in #518.
Summary
Fixes the non-stream half of the OpenRouter reasoning gap in AISIX-Cloud #648. OpenRouter (and some OpenAI-compatible aggregators) return a reasoning model's chain-of-thought at
message.reasoning(string) — not the DeepSeek-canonicalmessage.reasoning_content.OpenAiResponseMessagewas a closed struct deserializing onlyreasoning_content, somessage.reasoningwas dropped at decode. A reasoning model that emits its whole answer as reasoning (e.g.z-ai/glm-4.6via OpenRouter) then reached the customer with bothcontentANDreasoning_contentempty.Evidence (AISIX-Cloud real-chain e2e): the z-ai non-stream call returns
status=200, completion_tokens=230— the upstream generated output — yet the customer saw emptycontent+reasoning_content. Consistent across runs; not model nondeterminism.The streaming path already handles OpenRouter's
delta.reasoningvia the per-keyreasoning_fieldoverride — but that lift is streaming/delta-only (extract_reasoning_fieldhard-rejects non-deltapaths), so the non-stream response had no path to surface it.Change (auto-normalize — no per-key config required)
crates/aisix-provider-openai/src/wire.rs:message.reasoningonOpenAiResponseMessage(was dropped at deserialize).response_into_chat_response, lift it into the canonicalreasoning_contentextraslot when the canonicalreasoning_contentis absent/empty. DeepSeek-canonicalreasoning_contenttakes precedence when both are present; an emptyreasoningadds no noise field (same skip-empty rule as guardrails: streaming output moderation forwards content live before check_output (pre-P2 leak) #466). Symmetric with the existing non-streamreasoning_contentcapture.Scope / blast radius (this is a global OpenAI-wire change, not OpenRouter-gated)
OpenAiResponseMessageis the shared OpenAI chat-completions response type for the whole workspace — it's also whataisix-provider-azure-openaiand any future OpenAI-compatible bridge parse responses through (see the module doc onpubvisibility). So this normalization is not keyed to OpenRouter: any upstream routed through the OpenAI wire that returns a non-emptymessage.reasoning(and no/emptymessage.reasoning_content) will now have it surfaced as canonicalreasoning_content.This is intended —
reasoningis the de-facto field for OpenAI-compatible aggregators — and is safe because:reasoning_content);reasoningis empty/absent (serdedefault), so OpenAI / Azure-OpenAI / DeepSeek-compat upstreams that never emitreasoningare byte-for-byte unaffected.The blast radius is "OpenAI-compatible upstreams that emit
message.reasoning", which today is OpenRouter-class aggregators; calling it out so reviewers don't read it as an OpenRouter-only branch.Test plan
wire.rs: (1) OpenRouterreasoning→canonical; (2) canonical-reasoning_content-wins precedence; (3) empty-reasoning-adds-no-field; (4) empty-canonical-reasoning_content-falls-through-to-reasoning— the discriminating guard exercising the real precedence logic (reasoning_content.filter(!empty).or(reasoning)). Verified tests (1) and (4) FAIL against the pre-fix canonical-only code and pass with the fix (genuine regression guards, not non-discriminating).cargo clippy -p aisix-provider-openai+cargo fmt --checkclean.aisix-provider-azure-openai,aisix-proxy) build.Streaming parity (follow-up, out of scope here)
This PR makes the non-stream path auto-normalize with no operator config. Bringing the streaming path to the same no-config parity (default the
delta.reasoninglift for the OpenRouter adapter so it works without a per-keyreasoning_fieldoverride) is tracked in #502.Closes the non-stream half of api7/AISIX-Cloud#648. Once this lands in the
:devimage, the AISIX-Cloudopenrouter-longtailreal-chain z-ai non-stream leg goes green without the held-back workaround.Summary by CodeRabbit
Release Notes