Skip to content

fix(router): use LiteLLM master key in responses_api and default backend_conf in execute_proxy#384

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sheepdestroyer merged 4 commits into from
Aug 5, 2026
Merged

fix(router): use LiteLLM master key in responses_api and default backend_conf in execute_proxy #384

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
aa84644
fix(router): use LiteLLM master key in responses_api and default back…
Jul 26, 2026
d6d4e27
Merge branch 'master' into fix/responses-api-auth-and-execute-proxy
sheepdestroyer Aug 5, 2026
3e774f8
Merge branch 'master' into fix/responses-api-auth-and-execute-proxy
sheepdestroyer Aug 5, 2026
6d34d83
Merge branch 'master' into fix/responses-api-auth-and-execute-proxy
sheepdestroyer Aug 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 6 additions & 9 deletions router/main.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2125,12 +2125,8 @@ async def responses_api(request: Request):
body_to_send["model"] = target_model

litellm_key = os.getenv("LITELLM_MASTER_KEY")
auth_header = request.headers.get("Authorization")
if not auth_header or not auth_header.startswith("Bearer "):
auth_header = f"Bearer {litellm_key}"

headers = {
"Authorization": auth_header,
"Authorization": f"Bearer {litellm_key}",
"Content-Type": request.headers.get("content-type", "application/json"),
Comment on lines 2127 to 2130

@coderabbitaicoderabbitaiBotJul 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n -C 6 'responses_api|/responses|Depends\(|middleware|LITELLM_MASTER_KEY' router/main.py

Repository: sheepdestroyer/LLM-Routing

Length of output: 5601

🏁 Script executed:

#!/bin/bashset -euo pipefail
echo"== file size =="
wc -l router/main.py
echo"== app declaration and decorators/top-level routes within first 130 lines =="
sed -n '1,140p' router/main.py
echo"== relevant responses_api section =="
sed -n '2045,2185p' router/main.py
echo"== occurrences of dependencies/decorated handlers/routes =="
python3 - <<'PY'from pathlib import Pathp = Path("router/main.py")text = p.read_text()for i,line in enumerate(text.splitlines(),1): if "`@app`." in line or line.startswith("async def responses_api") or "Depends(" in line: start=max(1,i-8); end=min(len(text.splitlines()), i+8) print(f"\n--- around line {i} ---") for n in range(start,end+1): print(f"{n:4}: {text.splitlines()[n-1]}")PYecho"== route names/paths =="
python3 - <<'PY'from pathlib import Pathtext = Path("router/main.py").read_text()for i,line in enumerate(text.splitlines(),1): if ("`@app.api_route`" in line or "`@app.post`" in line or "`@app.get`" in line or "`@app.add_route`" in line): # show next line as route handler name/line print(f"{i}: {line.strip()}") if i < len(text.splitlines()): print(f"{i+1}: {text.splitlines()[i].strip()}")PY

Repository: sheepdestroyer/LLM-Routing

Length of output: 23590

Add client authentication before proxying Responses API requests.

responses_api(request: Request) is decorated by @app.api_route("/v1/responses") / @app.api_route("/responses") without a dependency, then forwards LITELLM_MASTER_KEY as upstream authorization. Add and enforce a client-auth dependency for these routes, or reject requests without a valid Authorization header, to avoid exposing the LiteLLM master-key quota to any reachable caller.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@router/main.py` around lines 2127 - 2130, The responses_api route currently
forwards the server’s LITELLM_MASTER_KEY without authenticating callers. Update
responses_api and its /v1/responses and /responses route declarations to enforce
the existing client-auth dependency, or validate and reject requests lacking a
valid Authorization header before proxying; preserve the upstream master-key
authorization only after client authentication succeeds.

}

Expand DownExpand Up@@ -2828,10 +2824,11 @@ async def execute_proxy(model_name: str):
# Resolve backend connection parameters
backend_conf = backends.get(model_name)
if not backend_conf:
logger.error(f"Backend '{model_name}' not found in configuration backends.")
raise HTTPException(
status_code=500, detail=f"Backend {model_name} misconfigured"
)
logger.info(f"Backend '{model_name}' not found in backends mapping, defaulting to LiteLLM proxy")
backend_conf = {
"api_base": f"{LITELLM_URL}/v1",
"api_key": "DYNAMIC_LITELLM_MASTER_KEY_PLACEHOLDER",
}

backend_api_base = backend_conf["api_base"]
backend_api_key = backend_conf["api_key"]
Expand Down