feat: add LiteLLM as LLM provider

[RheagalFire]

Summary

Adds LiteLLM as a new LLM provider, giving Open-Interface users access to 100+ LLM providers (Anthropic, AWS Bedrock, Vertex AI, Azure, Groq, Ollama, Cohere, Mistral, Together, Fireworks, etc.) through the LiteLLM Python SDK.

Uses litellm.completion() directly with drop_params=True. Provider API keys are read from environment variables (e.g. ANTHROPIC_API_KEY, AZURE_AI_API_KEY).

Changes

• app/models/litellm_model.py -- new LiteLLMModel extending Model, calls litellm.completion() with vision support (base64 screenshots)
• app/models/factory.py -- route models with provider prefixes (anthropic/, bedrock/, vertex_ai/, etc.) to LiteLLMModel
• requirements.txt -- add litellm>=1.80.0,<1.87.0

No UI changes -- users select Custom in settings, type the LiteLLM model string (e.g. anthropic/claude-sonnet-4-20250514), and the factory routes it automatically based on the provider/ prefix.

Live E2E (Azure Foundry -> Claude Sonnet 4.6, through actual codebase call path)

Tested using the same litellm.completion() call pattern from LiteLLMModel.send_message_to_llm():

# From app/models/litellm_model.py -- send_message_to_llm()kwargs= {
'model': self.model_name,
'messages': [
{
'role': 'user',
'content': message,
}
],
'max_tokens': 800,
'drop_params': True,
}
ifself.api_key:
kwargs['api_key'] =self.api_keyifself.base_url:
kwargs['api_base'] =self.base_urlresponse=litellm.completion(**kwargs)

Model: claude-sonnet-4-6
Response: pong
Tokens: 13 in, 5 out

Example usage

Select Custom in the settings UI, enter a LiteLLM model string in the provider/model format, and set your provider API key either in the UI's API Key field or as an environment variable:

• anthropic/claude-sonnet-4-20250514 -- Anthropic (reads ANTHROPIC_API_KEY from env)
• bedrock/anthropic.claude-sonnet-4-20250514-v1:0 -- AWS Bedrock (reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
• vertex_ai/gemini-2.5-flash -- Google Vertex (reads GOOGLE_APPLICATION_CREDENTIALS)
• azure_ai/claude-sonnet-4-6 -- Azure AI Foundry (reads AZURE_AI_API_KEY, AZURE_AI_API_BASE)
• groq/llama-3.3-70b-versatile -- Groq (reads GROQ_API_KEY)
• and so on

importlitellm# Provider API keys are read from environment variables# e.g. ANTHROPIC_API_KEY, OPENAI_API_KEY, AZURE_AI_API_KEY, etc.response=litellm.completion(
model="anthropic/claude-sonnet-4-20250514",
messages=[{"role": "user", "content": "Hello from Open-Interface!"}],
drop_params=True,
)
print(response.choices[0].message.content)

Risk / Compatibility

• Additive only -- no existing providers modified
• LiteLLMModel follows the same pattern as GPT4v (chat completions with vision)
• Models without a recognized provider prefix still fall through to the existing GPT4v fallback
• Default OpenAI base_url from settings is filtered out so it doesn't override LiteLLM's own provider routing

[RheagalFire]

cc @AmberSahdev

[RheagalFire]

Summary

Adds LiteLLM as a new LLM provider, giving Open-Interface users access to 100+ LLM providers (Anthropic, AWS Bedrock, Vertex AI, Azure, Groq, Ollama, Cohere, Mistral, Together, Fireworks, etc.) through the LiteLLM Python SDK.

Uses litellm.completion() directly with drop_params=True. Provider API keys are read from environment variables (e.g. ANTHROPIC_API_KEY, AZURE_AI_API_KEY).

Changes

• app/models/litellm_model.py -- new LiteLLMModel extending Model, calls litellm.completion() with vision support (base64 screenshots)
• app/models/factory.py -- route models with provider prefixes (anthropic/, bedrock/, vertex_ai/, etc.) to LiteLLMModel
• requirements.txt -- add litellm>=1.80.0,<1.87.0

No UI changes -- users select Custom in settings, type the LiteLLM model string (e.g. anthropic/claude-sonnet-4-20250514), and the factory routes it automatically based on the provider/ prefix.

Live E2E (Azure Foundry -> Claude Sonnet 4.6, through actual codebase call path)

Tested using the same litellm.completion() call pattern from LiteLLMModel.send_message_to_llm():

# From app/models/litellm_model.py -- send_message_to_llm()kwargs= {
'model': self.model_name,
'messages': [
{
'role': 'user',
'content': message,
}
],
'max_tokens': 800,
'drop_params': True,
}
ifself.api_key:
kwargs['api_key'] =self.api_keyifself.base_url:
kwargs['api_base'] =self.base_urlresponse=litellm.completion(**kwargs)

Model: claude-sonnet-4-6
Response: pong
Tokens: 13 in, 5 out

Example usage

Select Custom in the settings UI, enter a LiteLLM model string in the provider/model format, and set your provider API key either in the UI's API Key field or as an environment variable:

• anthropic/claude-sonnet-4-20250514 -- Anthropic (reads ANTHROPIC_API_KEY from env)
• bedrock/anthropic.claude-sonnet-4-20250514-v1:0 -- AWS Bedrock (reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
• vertex_ai/gemini-2.5-flash -- Google Vertex (reads GOOGLE_APPLICATION_CREDENTIALS)
• azure_ai/claude-sonnet-4-6 -- Azure AI Foundry (reads AZURE_AI_API_KEY, AZURE_AI_API_BASE)
• groq/llama-3.3-70b-versatile -- Groq (reads GROQ_API_KEY)
• and so on

importlitellm# Provider API keys are read from environment variables# e.g. ANTHROPIC_API_KEY, OPENAI_API_KEY, AZURE_AI_API_KEY, etc.response=litellm.completion(
model="anthropic/claude-sonnet-4-20250514",
messages=[{"role": "user", "content": "Hello from Open-Interface!"}],
drop_params=True,
)
print(response.choices[0].message.content)

Risk / Compatibility

• Additive only -- no existing providers modified
• LiteLLMModel follows the same pattern as GPT4v (chat completions with vision)
• Models without a recognized provider prefix still fall through to the existing GPT4v fallback
• Default OpenAI base_url from settings is filtered out so it doesn't override LiteLLM's own provider routing

[RheagalFire]

cc @AmberSahdev

[RheagalFire]

Summary

Adds LiteLLM as a new LLM provider, giving Open-Interface users access to 100+ LLM providers (Anthropic, AWS Bedrock, Vertex AI, Azure, Groq, Ollama, Cohere, Mistral, Together, Fireworks, etc.) through the LiteLLM Python SDK.

Uses litellm.completion() directly with drop_params=True. Provider API keys are read from environment variables (e.g. ANTHROPIC_API_KEY, AZURE_AI_API_KEY).

Changes

• app/models/litellm_model.py -- new LiteLLMModel extending Model, calls litellm.completion() with vision support (base64 screenshots)
• app/models/factory.py -- route models with provider prefixes (anthropic/, bedrock/, vertex_ai/, etc.) to LiteLLMModel
• requirements.txt -- add litellm>=1.80.0,<1.87.0

No UI changes -- users select Custom in settings, type the LiteLLM model string (e.g. anthropic/claude-sonnet-4-20250514), and the factory routes it automatically based on the provider/ prefix.

Live E2E (Azure Foundry -> Claude Sonnet 4.6, through actual codebase call path)

Tested using the same litellm.completion() call pattern from LiteLLMModel.send_message_to_llm():

# From app/models/litellm_model.py -- send_message_to_llm()kwargs= {
'model': self.model_name,
'messages': [
{
'role': 'user',
'content': message,
}
],
'max_tokens': 800,
'drop_params': True,
}
ifself.api_key:
kwargs['api_key'] =self.api_keyifself.base_url:
kwargs['api_base'] =self.base_urlresponse=litellm.completion(**kwargs)

Model: claude-sonnet-4-6
Response: pong
Tokens: 13 in, 5 out

Example usage

Select Custom in the settings UI, enter a LiteLLM model string in the provider/model format, and set your provider API key either in the UI's API Key field or as an environment variable:

• anthropic/claude-sonnet-4-20250514 -- Anthropic (reads ANTHROPIC_API_KEY from env)
• bedrock/anthropic.claude-sonnet-4-20250514-v1:0 -- AWS Bedrock (reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
• vertex_ai/gemini-2.5-flash -- Google Vertex (reads GOOGLE_APPLICATION_CREDENTIALS)
• azure_ai/claude-sonnet-4-6 -- Azure AI Foundry (reads AZURE_AI_API_KEY, AZURE_AI_API_BASE)
• groq/llama-3.3-70b-versatile -- Groq (reads GROQ_API_KEY)
• and so on

importlitellm# Provider API keys are read from environment variables# e.g. ANTHROPIC_API_KEY, OPENAI_API_KEY, AZURE_AI_API_KEY, etc.response=litellm.completion(
model="anthropic/claude-sonnet-4-20250514",
messages=[{"role": "user", "content": "Hello from Open-Interface!"}],
drop_params=True,
)
print(response.choices[0].message.content)

Risk / Compatibility

• Additive only -- no existing providers modified
• LiteLLMModel follows the same pattern as GPT4v (chat completions with vision)
• Models without a recognized provider prefix still fall through to the existing GPT4v fallback
• Default OpenAI base_url from settings is filtered out so it doesn't override LiteLLM's own provider routing

[RheagalFire]

cc @AmberSahdev

[RheagalFire]

Summary

Adds LiteLLM as a new LLM provider, giving Open-Interface users access to 100+ LLM providers (Anthropic, AWS Bedrock, Vertex AI, Azure, Groq, Ollama, Cohere, Mistral, Together, Fireworks, etc.) through the LiteLLM Python SDK.

Uses litellm.completion() directly with drop_params=True. Provider API keys are read from environment variables (e.g. ANTHROPIC_API_KEY, AZURE_AI_API_KEY).

Changes

• app/models/litellm_model.py -- new LiteLLMModel extending Model, calls litellm.completion() with vision support (base64 screenshots)
• app/models/factory.py -- route models with provider prefixes (anthropic/, bedrock/, vertex_ai/, etc.) to LiteLLMModel
• requirements.txt -- add litellm>=1.80.0,<1.87.0

No UI changes -- users select Custom in settings, type the LiteLLM model string (e.g. anthropic/claude-sonnet-4-20250514), and the factory routes it automatically based on the provider/ prefix.

Live E2E (Azure Foundry -> Claude Sonnet 4.6, through actual codebase call path)

Tested using the same litellm.completion() call pattern from LiteLLMModel.send_message_to_llm():

# From app/models/litellm_model.py -- send_message_to_llm()kwargs= {
'model': self.model_name,
'messages': [
{
'role': 'user',
'content': message,
}
],
'max_tokens': 800,
'drop_params': True,
}
ifself.api_key:
kwargs['api_key'] =self.api_keyifself.base_url:
kwargs['api_base'] =self.base_urlresponse=litellm.completion(**kwargs)

Model: claude-sonnet-4-6
Response: pong
Tokens: 13 in, 5 out

Example usage

Select Custom in the settings UI, enter a LiteLLM model string in the provider/model format, and set your provider API key either in the UI's API Key field or as an environment variable:

• anthropic/claude-sonnet-4-20250514 -- Anthropic (reads ANTHROPIC_API_KEY from env)
• bedrock/anthropic.claude-sonnet-4-20250514-v1:0 -- AWS Bedrock (reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
• vertex_ai/gemini-2.5-flash -- Google Vertex (reads GOOGLE_APPLICATION_CREDENTIALS)
• azure_ai/claude-sonnet-4-6 -- Azure AI Foundry (reads AZURE_AI_API_KEY, AZURE_AI_API_BASE)
• groq/llama-3.3-70b-versatile -- Groq (reads GROQ_API_KEY)
• and so on

importlitellm# Provider API keys are read from environment variables# e.g. ANTHROPIC_API_KEY, OPENAI_API_KEY, AZURE_AI_API_KEY, etc.response=litellm.completion(
model="anthropic/claude-sonnet-4-20250514",
messages=[{"role": "user", "content": "Hello from Open-Interface!"}],
drop_params=True,
)
print(response.choices[0].message.content)

Risk / Compatibility

• Additive only -- no existing providers modified
• LiteLLMModel follows the same pattern as GPT4v (chat completions with vision)
• Models without a recognized provider prefix still fall through to the existing GPT4v fallback
• Default OpenAI base_url from settings is filtered out so it doesn't override LiteLLM's own provider routing

[RheagalFire]

cc @AmberSahdev

[RheagalFire]

Summary

Adds LiteLLM as a new LLM provider, giving Open-Interface users access to 100+ LLM providers (Anthropic, AWS Bedrock, Vertex AI, Azure, Groq, Ollama, Cohere, Mistral, Together, Fireworks, etc.) through the LiteLLM Python SDK.

Uses litellm.completion() directly with drop_params=True. Provider API keys are read from environment variables (e.g. ANTHROPIC_API_KEY, AZURE_AI_API_KEY).

Changes

• app/models/litellm_model.py -- new LiteLLMModel extending Model, calls litellm.completion() with vision support (base64 screenshots)
• app/models/factory.py -- route models with provider prefixes (anthropic/, bedrock/, vertex_ai/, etc.) to LiteLLMModel
• requirements.txt -- add litellm>=1.80.0,<1.87.0

No UI changes -- users select Custom in settings, type the LiteLLM model string (e.g. anthropic/claude-sonnet-4-20250514), and the factory routes it automatically based on the provider/ prefix.

Live E2E (Azure Foundry -> Claude Sonnet 4.6, through actual codebase call path)

Tested using the same litellm.completion() call pattern from LiteLLMModel.send_message_to_llm():

# From app/models/litellm_model.py -- send_message_to_llm()kwargs= {
'model': self.model_name,
'messages': [
{
'role': 'user',
'content': message,
}
],
'max_tokens': 800,
'drop_params': True,
}
ifself.api_key:
kwargs['api_key'] =self.api_keyifself.base_url:
kwargs['api_base'] =self.base_urlresponse=litellm.completion(**kwargs)

Model: claude-sonnet-4-6
Response: pong
Tokens: 13 in, 5 out

Example usage

Select Custom in the settings UI, enter a LiteLLM model string in the provider/model format, and set your provider API key either in the UI's API Key field or as an environment variable:

• anthropic/claude-sonnet-4-20250514 -- Anthropic (reads ANTHROPIC_API_KEY from env)
• bedrock/anthropic.claude-sonnet-4-20250514-v1:0 -- AWS Bedrock (reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
• vertex_ai/gemini-2.5-flash -- Google Vertex (reads GOOGLE_APPLICATION_CREDENTIALS)
• azure_ai/claude-sonnet-4-6 -- Azure AI Foundry (reads AZURE_AI_API_KEY, AZURE_AI_API_BASE)
• groq/llama-3.3-70b-versatile -- Groq (reads GROQ_API_KEY)
• and so on

importlitellm# Provider API keys are read from environment variables# e.g. ANTHROPIC_API_KEY, OPENAI_API_KEY, AZURE_AI_API_KEY, etc.response=litellm.completion(
model="anthropic/claude-sonnet-4-20250514",
messages=[{"role": "user", "content": "Hello from Open-Interface!"}],
drop_params=True,
)
print(response.choices[0].message.content)

Risk / Compatibility

• Additive only -- no existing providers modified
• LiteLLMModel follows the same pattern as GPT4v (chat completions with vision)
• Models without a recognized provider prefix still fall through to the existing GPT4v fallback
• Default OpenAI base_url from settings is filtered out so it doesn't override LiteLLM's own provider routing

[RheagalFire]

cc @AmberSahdev

[RheagalFire]

Summary

Adds LiteLLM as a new LLM provider, giving Open-Interface users access to 100+ LLM providers (Anthropic, AWS Bedrock, Vertex AI, Azure, Groq, Ollama, Cohere, Mistral, Together, Fireworks, etc.) through the LiteLLM Python SDK.

Uses litellm.completion() directly with drop_params=True. Provider API keys are read from environment variables (e.g. ANTHROPIC_API_KEY, AZURE_AI_API_KEY).

Changes

• app/models/litellm_model.py -- new LiteLLMModel extending Model, calls litellm.completion() with vision support (base64 screenshots)
• app/models/factory.py -- route models with provider prefixes (anthropic/, bedrock/, vertex_ai/, etc.) to LiteLLMModel
• requirements.txt -- add litellm>=1.80.0,<1.87.0

No UI changes -- users select Custom in settings, type the LiteLLM model string (e.g. anthropic/claude-sonnet-4-20250514), and the factory routes it automatically based on the provider/ prefix.

Live E2E (Azure Foundry -> Claude Sonnet 4.6, through actual codebase call path)

Tested using the same litellm.completion() call pattern from LiteLLMModel.send_message_to_llm():

# From app/models/litellm_model.py -- send_message_to_llm()kwargs= {
'model': self.model_name,
'messages': [
{
'role': 'user',
'content': message,
}
],
'max_tokens': 800,
'drop_params': True,
}
ifself.api_key:
kwargs['api_key'] =self.api_keyifself.base_url:
kwargs['api_base'] =self.base_urlresponse=litellm.completion(**kwargs)

Model: claude-sonnet-4-6
Response: pong
Tokens: 13 in, 5 out

Example usage

Select Custom in the settings UI, enter a LiteLLM model string in the provider/model format, and set your provider API key either in the UI's API Key field or as an environment variable:

• anthropic/claude-sonnet-4-20250514 -- Anthropic (reads ANTHROPIC_API_KEY from env)
• bedrock/anthropic.claude-sonnet-4-20250514-v1:0 -- AWS Bedrock (reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
• vertex_ai/gemini-2.5-flash -- Google Vertex (reads GOOGLE_APPLICATION_CREDENTIALS)
• azure_ai/claude-sonnet-4-6 -- Azure AI Foundry (reads AZURE_AI_API_KEY, AZURE_AI_API_BASE)
• groq/llama-3.3-70b-versatile -- Groq (reads GROQ_API_KEY)
• and so on

importlitellm# Provider API keys are read from environment variables# e.g. ANTHROPIC_API_KEY, OPENAI_API_KEY, AZURE_AI_API_KEY, etc.response=litellm.completion(
model="anthropic/claude-sonnet-4-20250514",
messages=[{"role": "user", "content": "Hello from Open-Interface!"}],
drop_params=True,
)
print(response.choices[0].message.content)

Risk / Compatibility

• Additive only -- no existing providers modified
• LiteLLMModel follows the same pattern as GPT4v (chat completions with vision)
• Models without a recognized provider prefix still fall through to the existing GPT4v fallback
• Default OpenAI base_url from settings is filtered out so it doesn't override LiteLLM's own provider routing

[RheagalFire]

cc @AmberSahdev

[RheagalFire]

Summary

Adds LiteLLM as a new LLM provider, giving Open-Interface users access to 100+ LLM providers (Anthropic, AWS Bedrock, Vertex AI, Azure, Groq, Ollama, Cohere, Mistral, Together, Fireworks, etc.) through the LiteLLM Python SDK.

Uses litellm.completion() directly with drop_params=True. Provider API keys are read from environment variables (e.g. ANTHROPIC_API_KEY, AZURE_AI_API_KEY).

Changes

• app/models/litellm_model.py -- new LiteLLMModel extending Model, calls litellm.completion() with vision support (base64 screenshots)
• app/models/factory.py -- route models with provider prefixes (anthropic/, bedrock/, vertex_ai/, etc.) to LiteLLMModel
• requirements.txt -- add litellm>=1.80.0,<1.87.0

No UI changes -- users select Custom in settings, type the LiteLLM model string (e.g. anthropic/claude-sonnet-4-20250514), and the factory routes it automatically based on the provider/ prefix.

Live E2E (Azure Foundry -> Claude Sonnet 4.6, through actual codebase call path)

Tested using the same litellm.completion() call pattern from LiteLLMModel.send_message_to_llm():

# From app/models/litellm_model.py -- send_message_to_llm()kwargs= {
'model': self.model_name,
'messages': [
{
'role': 'user',
'content': message,
}
],
'max_tokens': 800,
'drop_params': True,
}
ifself.api_key:
kwargs['api_key'] =self.api_keyifself.base_url:
kwargs['api_base'] =self.base_urlresponse=litellm.completion(**kwargs)

Model: claude-sonnet-4-6
Response: pong
Tokens: 13 in, 5 out

Example usage

Select Custom in the settings UI, enter a LiteLLM model string in the provider/model format, and set your provider API key either in the UI's API Key field or as an environment variable:

• anthropic/claude-sonnet-4-20250514 -- Anthropic (reads ANTHROPIC_API_KEY from env)
• bedrock/anthropic.claude-sonnet-4-20250514-v1:0 -- AWS Bedrock (reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
• vertex_ai/gemini-2.5-flash -- Google Vertex (reads GOOGLE_APPLICATION_CREDENTIALS)
• azure_ai/claude-sonnet-4-6 -- Azure AI Foundry (reads AZURE_AI_API_KEY, AZURE_AI_API_BASE)
• groq/llama-3.3-70b-versatile -- Groq (reads GROQ_API_KEY)
• and so on

importlitellm# Provider API keys are read from environment variables# e.g. ANTHROPIC_API_KEY, OPENAI_API_KEY, AZURE_AI_API_KEY, etc.response=litellm.completion(
model="anthropic/claude-sonnet-4-20250514",
messages=[{"role": "user", "content": "Hello from Open-Interface!"}],
drop_params=True,
)
print(response.choices[0].message.content)

Risk / Compatibility

• Additive only -- no existing providers modified
• LiteLLMModel follows the same pattern as GPT4v (chat completions with vision)
• Models without a recognized provider prefix still fall through to the existing GPT4v fallback
• Default OpenAI base_url from settings is filtered out so it doesn't override LiteLLM's own provider routing

[RheagalFire]

cc @AmberSahdev

[RheagalFire]

Summary

Adds LiteLLM as a new LLM provider, giving Open-Interface users access to 100+ LLM providers (Anthropic, AWS Bedrock, Vertex AI, Azure, Groq, Ollama, Cohere, Mistral, Together, Fireworks, etc.) through the LiteLLM Python SDK.

Uses litellm.completion() directly with drop_params=True. Provider API keys are read from environment variables (e.g. ANTHROPIC_API_KEY, AZURE_AI_API_KEY).

Changes

• app/models/litellm_model.py -- new LiteLLMModel extending Model, calls litellm.completion() with vision support (base64 screenshots)
• app/models/factory.py -- route models with provider prefixes (anthropic/, bedrock/, vertex_ai/, etc.) to LiteLLMModel
• requirements.txt -- add litellm>=1.80.0,<1.87.0

No UI changes -- users select Custom in settings, type the LiteLLM model string (e.g. anthropic/claude-sonnet-4-20250514), and the factory routes it automatically based on the provider/ prefix.

Live E2E (Azure Foundry -> Claude Sonnet 4.6, through actual codebase call path)

Tested using the same litellm.completion() call pattern from LiteLLMModel.send_message_to_llm():

# From app/models/litellm_model.py -- send_message_to_llm()kwargs= {
'model': self.model_name,
'messages': [
{
'role': 'user',
'content': message,
}
],
'max_tokens': 800,
'drop_params': True,
}
ifself.api_key:
kwargs['api_key'] =self.api_keyifself.base_url:
kwargs['api_base'] =self.base_urlresponse=litellm.completion(**kwargs)

Model: claude-sonnet-4-6
Response: pong
Tokens: 13 in, 5 out

Example usage

Select Custom in the settings UI, enter a LiteLLM model string in the provider/model format, and set your provider API key either in the UI's API Key field or as an environment variable:

• anthropic/claude-sonnet-4-20250514 -- Anthropic (reads ANTHROPIC_API_KEY from env)
• bedrock/anthropic.claude-sonnet-4-20250514-v1:0 -- AWS Bedrock (reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
• vertex_ai/gemini-2.5-flash -- Google Vertex (reads GOOGLE_APPLICATION_CREDENTIALS)
• azure_ai/claude-sonnet-4-6 -- Azure AI Foundry (reads AZURE_AI_API_KEY, AZURE_AI_API_BASE)
• groq/llama-3.3-70b-versatile -- Groq (reads GROQ_API_KEY)
• and so on

importlitellm# Provider API keys are read from environment variables# e.g. ANTHROPIC_API_KEY, OPENAI_API_KEY, AZURE_AI_API_KEY, etc.response=litellm.completion(
model="anthropic/claude-sonnet-4-20250514",
messages=[{"role": "user", "content": "Hello from Open-Interface!"}],
drop_params=True,
)
print(response.choices[0].message.content)

Risk / Compatibility

• Additive only -- no existing providers modified
• LiteLLMModel follows the same pattern as GPT4v (chat completions with vision)
• Models without a recognized provider prefix still fall through to the existing GPT4v fallback
• Default OpenAI base_url from settings is filtered out so it doesn't override LiteLLM's own provider routing

[RheagalFire]

cc @AmberSahdev