Add new AI models, update docs by bubacoder · Pull Request #137 · bubacoder/infra

bubacoder · 2025-05-23T20:37:00Z

Summary by CodeRabbit

New Features
- Added support for new Anthropic models and updated Openrouter models in the AI model configuration.
- Introduced clearer grouping and comments for local models in the configuration.
Documentation
- Updated documentation to include tools for users with limited GPU resources.
- Added notes highlighting the coding capabilities of general-purpose proprietary models and referenced relevant leaderboards.

coderabbitai · 2025-05-23T20:37:05Z

Walkthrough

The pull request updates the LiteLLM configuration by adding new Anthropic models, modifying Openrouter model entries, and explicitly grouping local Ollama models. It also enhances documentation for open source models by adding resources for limited GPU environments and updating notes on coding-optimized models.

Changes

File(s)	Change Summary
docker/ai/litellm/config.yaml	Added new Anthropic models, updated Openrouter model names and keys, introduced explicit local model section with Ollama entries.
docs/ai/open-models.md	Added tools for limited GPU resources, updated notes on coding-optimized models, and included a TODO for best models update.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant LiteLLM Config
    participant Anthropic API
    participant Openrouter API
    participant Local Model

    User->>LiteLLM Config: Request model inference
    alt Anthropic model
        LiteLLM Config->>Anthropic API: Forward request with API key
        Anthropic API-->>LiteLLM Config: Return response
    else Openrouter model
        LiteLLM Config->>Openrouter API: Forward request with API key
        Openrouter API-->>LiteLLM Config: Return response
    else Local model
        LiteLLM Config->>Local Model: Invoke local inference
        Local Model-->>LiteLLM Config: Return response
    end
    LiteLLM Config-->>User: Return inference result

Possibly related PRs

bubacoder/infra#100: Modifies docker/ai/litellm/config.yaml by adding new model entries and reorganizing model configurations, directly related to expanding and updating LiteLLM model configurations.

Note

⚡️ AI Code Reviews for VS Code, Cursor, Windsurf

CodeRabbit now has a plugin for VS Code, Cursor and Windsurf. This brings AI code reviews directly in the code editor. Each commit is reviewed immediately, finding bugs before the PR is raised. Seamless context handoff to your AI code agent ensures that you can easily incorporate review feedback.
Learn more here.

📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 270ba21 and 7f86b79.

📒 Files selected for processing (2)

docker/ai/litellm/config.yaml (3 hunks)
docs/ai/open-models.md (2 hunks)

🚧 Files skipped from review as they are similar to previous changes (2)

docs/ai/open-models.md
docker/ai/litellm/config.yaml

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Explain this complex logic.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai explain this code block.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and explain its main purpose.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai generate docstrings to generate docstrings for this PR.
@coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

coderabbitai

Actionable comments posted: 2

🧹 Nitpick comments (5)

docs/ai/open-models.md (3)

12-14: Use a proper Markdown heading for the new section
The “Tools for limited GPU resources” block is inserted inline—consider converting it into a level-3 heading (### Tools for limited GPU resources) to match the surrounding structure and improve readability and navigation.

16-16: Convert the TODO into a tracked task
The inline TODO: Update the list of best models should either be resolved before merging or linked to a GitHub issue so it doesn’t get lost.

28-28: Format the coding-optimized note as an admonition
Instead of a plain sentence, consider using a Markdown quote block (> Note:) or an admonition to highlight that proprietary models now excel at coding. This will make the note more visible.

docker/ai/litellm/config.yaml (2)

24-38: Validate new Anthropic model entries and API key syntax
The new entries for claude-3-7-sonnet, claude-opus-4-0, and claude-sonnet-4-0 look structurally sound, but please:

Confirm the model strings (e.g., anthropic/claude-3-7-sonnet-latest) exactly match Anthropic’s published names.

Ensure "os.environ/ANTHROPIC_API_KEY" is correctly interpreted by Litellm (it may need alternate placeholder syntax or removal of quotes).

78-93: Review local model configuration and environment variables
The local models (ollama-local-phi and ollama-mac-mistral) are useful additions, but:

Double-check that LOCAL_OLLAMA_API_BASE vs REMOTE_OLLAMA_API_BASE align with your deployment and are documented in your README or environment setup.

Consider renaming ollama-mac-mistral to something like ollama-mistral-local for clarity and consistency.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 352ab13 and 270ba21.

📒 Files selected for processing (2)

docker/ai/litellm/config.yaml (3 hunks)
docs/ai/open-models.md (2 hunks)

🔇 Additional comments (1)

docker/ai/litellm/config.yaml (1)

43-47: Verify Openrouter model IDs and key handling
The updated gpt-4o entry and new o4-mini-high model should be cross-checked against Openrouter’s current naming conventions. Also confirm that the "os.environ/OPENROUTER_API_KEY" literal is parsed as an environment variable by Litellm’s loader.

Also applies to: 54-59

coderabbitai Bot reviewed May 23, 2025

View reviewed changes

Comment thread docker/ai/litellm/config.yaml Outdated

Comment thread docker/ai/litellm/config.yaml Outdated

bubacoder added 2 commits May 24, 2025 20:16

Add new Anthropic and OpenAI models

6d8b240

Add small model comparision, RAM calculator

7f86b79

bubacoder force-pushed the feature/llm branch from 270ba21 to 7f86b79 Compare May 24, 2025 20:17

bubacoder merged commit 57a6deb into main May 25, 2025

bubacoder deleted the feature/llm branch May 25, 2025 06:26

This was referenced Jul 18, 2025

Extend update-docs: verify and update relative links in the documentation #153

Merged

Update AI and VM documentation #157

Merged

coderabbitai Bot mentioned this pull request Aug 8, 2025

Configure OpenAI provider and GPT-5 models #174

Merged

coderabbitai Bot mentioned this pull request Oct 18, 2025

Update Anthropic models #213

Merged

coderabbitai Bot mentioned this pull request Dec 1, 2025

Upgrade LiteLLM, add new models #237

Merged

coderabbitai Bot mentioned this pull request Feb 27, 2026

Update LiteLLM models #288

Merged

coderabbitai Bot mentioned this pull request Apr 12, 2026

Update LiteLLM model configurations and Ollama pull script to use latest model versions #312

Merged

coderabbitai Bot mentioned this pull request Apr 26, 2026

Helper scripts #318

Merged

Navigation Menu

Add new AI models, update docs - #137

Add new AI models, update docs#137
bubacoder merged 2 commits into
mainfrom
feature/llm

bubacoder commented May 23, 2025 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented May 23, 2025 •

edited

Loading

Chat

Support

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

Documentation and Community

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

bubacoder commented May 23, 2025 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented May 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Possibly related PRs

Chat

Support

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

Documentation and Community

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

bubacoder commented May 23, 2025 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented May 23, 2025 •

edited

Loading