fix(dev): restore fallback capacity for canonical E2E chat verification (#345) - #468
Merged
Merged
Conversation
Contributor
Reviewer's GuideRestores and hardens environment-based configuration and verification for LiteLLM/OpenRouter routing, adds strict master-key and client auth validation, improves Langfuse tracing/session propagation, and extends tests/startup scripts to cover the new behavior and canonical endpoint derivation paths. Sequence diagram for responses_api client auth and master key validationsequenceDiagram
actor Client
participant Router as responses_api
participant Env as _validate_litellm_master_key
participant LiteLLM as litellm_proxy
Client->>Router: POST /responses (Authorization: Bearer client_token)
Router->>Router: validate Authorization header
Router->>Env: _validate_litellm_master_key()
Env-->>Router: master_key
Router->>LiteLLM: client.post /responses (Authorization: Bearer master_key)
LiteLLM-->>Router: response
Router-->>Client: routed response
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Contributor
Warning Review limit reached
Next review available in:117 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (10)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
There was a problem hiding this comment.
Hey - I've found 2 issues, and left some high level feedback:
- Avoid logging the raw LITELLM_MASTER_KEY value in
_validate_litellm_master_key, as the currentlogger.errorcall can expose secrets in logs; consider logging only that it is missing/invalid or a redacted version. - The client auth check in
responses_apionly validates the presence and shape of a Bearer token; if stronger authentication/authorization is expected, consider integrating actual token validation or clarifying that this endpoint is only gating on header presence.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments- Avoid logging the raw LITELLM_MASTER_KEY value in `_validate_litellm_master_key`, as the current `logger.error` call can expose secrets in logs; consider logging only that it is missing/invalid or a redacted version.
- The client auth check in `responses_api` only validates the presence and shape of a Bearer token; if stronger authentication/authorization is expected, consider integrating actual token validation or clarifying that this endpoint is only gating on header presence.
## Individual Comments### Comment 1
<locationpath="router/main.py"line_range="797-806" />
<code_context>
+}
++
+def _validate_litellm_master_key() -> str:
+ """Validate LITELLM_MASTER_KEY environment variable.
++ Returns:
+ The valid master key string.++ Raises:
+ HTTPException(500): If master key is missing, empty, or placeholder string.+ """
+ key = (os.getenv("LITELLM_MASTER_KEY") or "").strip()
+ if not key or key in _INVALID_MASTER_KEYS or "PLACEHOLDER" in key.upper():
+ logger.error(f"Invalid or missing LITELLM_MASTER_KEY: '{key}'")
</code_context>
<issue_to_address>
**🚨 issue (security):** Avoid logging the raw master key value to prevent leaking secrets.
The `logger.error` line logs the full `LITELLM_MASTER_KEY`, which risks exposing a production secret in logs. Please avoid logging the raw key—log only that it is invalid/missing, or mask it (e.g., show a short prefix or just its length) while preserving enough context for debugging.
</issue_to_address>
### Comment 2
<locationpath="router/main.py"line_range="2306-2310" />
<code_context>
when an auto model (e.g. llm-routing-auto-free) is requested, while supporting model aliases
(such as gpt-4o-mini, local-qwen-3.6-hass) and tool/streaming executions.
"""
+# Enforce client authentication+ auth_header = request.headers.get("Authorization") or request.headers.get("authorization")
+ if not auth_header or not auth_header.startswith("Bearer "):
+ raise HTTPException(status_code=401, detail="Missing or invalid Authorization header")+ client_token = auth_header[7:].strip()
+ if not client_token:
+ raise HTTPException(status_code=401, detail="Missing or invalid Authorization header")
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Authorization handling is case-sensitive and extracts a token that is never used.
`auth_header.startswith("Bearer ")` will reject headers with different casing or spacing (e.g. `bearer <token>`). Consider normalizing (e.g. lowercasing and/or splitting on whitespace) before checking to make the auth handling more robust. Also, `client_token` is extracted but never used; either pass it to downstream logic where needed or remove it to avoid confusion.
Suggested implementation:
```pythontry:
await _register_ollama_models_in_db(litellm_master_key)
exceptExceptionas e:
when an auto model (e.g. llm-routing-auto-free) is requested, while supporting model aliases
(such as gpt-4o-mini, local-qwen-3.6-hass) and tool/streaming executions.
""" # Enforce client authentication auth_header = request.headers.get("Authorization") or request.headers.get("authorization") if not auth_header: raise HTTPException(status_code=401, detail="Missing or invalid Authorization header") parts = auth_header.split() if len(parts) != 2 or parts[0].lower() != "bearer" or not parts[1]: raise HTTPException(status_code=401, detail="Missing or invalid Authorization header") # Store the normalized client token for downstream use request.state.client_token = parts[1] try:```1. Anywhere downstream that needs the client token should read it from `request.state.client_token` instead of re-parsing headers.2. If your router uses dependency injection (e.g. FastAPI dependencies) for auth, consider integrating this parsing into a shared dependency to avoid duplication.</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Comment on lines
+797
to
+806
| def _validate_litellm_master_key() -> str: | ||
| """Validate LITELLM_MASTER_KEY environment variable. | ||
| Returns: | ||
| The valid master key string. | ||
| Raises: | ||
| HTTPException(500): If master key is missing, empty, or placeholder string. | ||
| """ | ||
| key = (os.getenv("LITELLM_MASTER_KEY") or "").strip() |
Contributor
There was a problem hiding this comment.
🚨 issue (security): Avoid logging the raw master key value to prevent leaking secrets.
The logger.error line logs the full LITELLM_MASTER_KEY, which risks exposing a production secret in logs. Please avoid logging the raw key—log only that it is invalid/missing, or mask it (e.g., show a short prefix or just its length) while preserving enough context for debugging.
Comment on lines
+2306
to
+2310
| # Enforce client authentication | ||
| auth_header = request.headers.get("Authorization") or request.headers.get("authorization") | ||
| if not auth_header or not auth_header.startswith("Bearer "): | ||
| raise HTTPException(status_code=401, detail="Missing or invalid Authorization header") | ||
| client_token = auth_header[7:].strip() |
Contributor
There was a problem hiding this comment.
suggestion (bug_risk): Authorization handling is case-sensitive and extracts a token that is never used.
auth_header.startswith("Bearer ") will reject headers with different casing or spacing (e.g. bearer <token>). Consider normalizing (e.g. lowercasing and/or splitting on whitespace) before checking to make the auth handling more robust. Also, client_token is extracted but never used; either pass it to downstream logic where needed or remove it to avoid confusion.
Suggested implementation:
try:
await_register_ollama_models_in_db(litellm_master_key)
exceptExceptionase:
whenanautomodel (e.g. llm-routing-auto-free) isrequested, whilesupportingmodelaliases
(suchasgpt-4o-mini, local-qwen-3.6-hass) andtool/streamingexecutions.
"""
# Enforce client authentication
auth_header = request.headers.get("Authorization") or request.headers.get("authorization")
if not auth_header:
raise HTTPException(status_code=401, detail="Missing or invalid Authorization header")
parts = auth_header.split()
if len(parts) != 2 or parts[0].lower() != "bearer" or not parts[1]:
raise HTTPException(status_code=401, detail="Missing or invalid Authorization header")
# Store the normalized client token for downstream userequest.state.client_token=parts[1]
try:- Anywhere downstream that needs the client token should read it from
request.state.client_tokeninstead of re-parsing headers. - If your router uses dependency injection (e.g. FastAPI dependencies) for auth, consider integrating this parsing into a shared dependency to avoid duplication.
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes#345
Summary by Sourcery
Harden routing and verification infrastructure around LiteLLM/OpenRouter and Langfuse, improving auth, key handling, and canonical endpoint behavior.
Bug Fixes:
Enhancements:
Tests: