fix(router): use LiteLLM master key in responses_api and default backend_conf in execute_proxy - #384
Conversation
…end_conf in execute_proxy
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
🧙 Sourcery is reviewing your pull request! Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Warning Review limit reached
Next review available in:51 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. 📝 WalkthroughWalkthroughThe proxy now always forwards the configured LiteLLM master key for ChangesProxy routing behavior
Estimated code review effort: 2 (Simple) | ~10 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
router/main.py (1)
2127-2130: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winFail fast when
LITELLM_MASTER_KEYis unavailable.
os.getenvcan produceBearer Nonefor Responses requests, while the new fallback can resolve to the literal placeholder. Validate the key at startup, or return a clear configuration error before proxying instead of producing opaque upstream authorization failures.Also applies to: 2826-2830
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@router/main.py` around lines 2127 - 2130, Validate LITELLM_MASTER_KEY before constructing the Authorization header in the Responses request path and the corresponding code at the additional occurrence. Fail fast with a clear configuration error when the key is unset or resolves to a placeholder, preventing requests from being proxied with invalid credentials; otherwise preserve the existing Bearer header behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@router/main.py`:
- Around line 2127-2130: The responses_api route currently forwards the server’s
LITELLM_MASTER_KEY without authenticating callers. Update responses_api and its
/v1/responses and /responses route declarations to enforce the existing
client-auth dependency, or validate and reject requests lacking a valid
Authorization header before proxying; preserve the upstream master-key
authorization only after client authentication succeeds.
---
Nitpick comments:
In `@router/main.py`:
- Around line 2127-2130: Validate LITELLM_MASTER_KEY before constructing the
Authorization header in the Responses request path and the corresponding code at
the additional occurrence. Fail fast with a clear configuration error when the
key is unset or resolves to a placeholder, preventing requests from being
proxied with invalid credentials; otherwise preserve the existing Bearer header
behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
| litellm_key = os.getenv("LITELLM_MASTER_KEY") | ||
| auth_header = request.headers.get("Authorization") | ||
| if not auth_header or not auth_header.startswith("Bearer "): | ||
| auth_header = f"Bearer {litellm_key}" | ||
| headers = { | ||
| "Authorization": auth_header, | ||
| "Authorization": f"Bearer {litellm_key}", | ||
| "Content-Type": request.headers.get("content-type", "application/json"), |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
rg -n -C 6 'responses_api|/responses|Depends\(|middleware|LITELLM_MASTER_KEY' router/main.pyRepository: sheepdestroyer/LLM-Routing
Length of output: 5601
🏁 Script executed:
#!/bin/bashset -euo pipefail
echo"== file size =="
wc -l router/main.py
echo"== app declaration and decorators/top-level routes within first 130 lines =="
sed -n '1,140p' router/main.py
echo"== relevant responses_api section =="
sed -n '2045,2185p' router/main.py
echo"== occurrences of dependencies/decorated handlers/routes =="
python3 - <<'PY'from pathlib import Pathp = Path("router/main.py")text = p.read_text()for i,line in enumerate(text.splitlines(),1): if "`@app`." in line or line.startswith("async def responses_api") or "Depends(" in line: start=max(1,i-8); end=min(len(text.splitlines()), i+8) print(f"\n--- around line {i} ---") for n in range(start,end+1): print(f"{n:4}: {text.splitlines()[n-1]}")PYecho"== route names/paths =="
python3 - <<'PY'from pathlib import Pathtext = Path("router/main.py").read_text()for i,line in enumerate(text.splitlines(),1): if ("`@app.api_route`" in line or "`@app.post`" in line or "`@app.get`" in line or "`@app.add_route`" in line): # show next line as route handler name/line print(f"{i}: {line.strip()}") if i < len(text.splitlines()): print(f"{i+1}: {text.splitlines()[i].strip()}")PYRepository: sheepdestroyer/LLM-Routing
Length of output: 23590
Add client authentication before proxying Responses API requests.
responses_api(request: Request) is decorated by @app.api_route("/v1/responses") / @app.api_route("/responses") without a dependency, then forwards LITELLM_MASTER_KEY as upstream authorization. Add and enforce a client-auth dependency for these routes, or reject requests without a valid Authorization header, to avoid exposing the LiteLLM master-key quota to any reachable caller.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@router/main.py` around lines 2127 - 2130, The responses_api route currently
forwards the server’s LITELLM_MASTER_KEY without authenticating callers. Update
responses_api and its /v1/responses and /responses route declarations to enforce
the existing client-auth dependency, or validate and reject requests lacking a
valid Authorization header before proxying; preserve the upstream master-key
authorization only after client authentication succeeds.
sheepdestroyer
commented
Aug 5, 2026
Linked to issue #407 for tracking follow-up security validation and dynamic OpenRouter model sync fixes. |
Fixes 2 issues discovered during dev endpoint verification:
Verified with (35/35 passed).
Summary by Sourcery
Use LiteLLM master key consistently for proxied responses and default unmapped backend models to the LiteLLM proxy.
Bug Fixes:
Summary by CodeRabbit