Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/adapters/openai-compatible.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -15,6 +15,7 @@ keywords:
- perplexity
- lm studio
- vllm
- litellm
- adapter
---

Expand DownExpand Up@@ -184,6 +185,29 @@ const ollama = openaiCompatible({

> Ollama also has a dedicated adapter, [`@tanstack/ai-ollama`](./ollama), which understands its native API. Use `openaiCompatible` only if you specifically want Ollama's OpenAI-compatible surface.

## LiteLLM Proxy

[LiteLLM](https://github.com/BerriAI/litellm) is a self-hosted gateway that exposes a single OpenAI Chat Completions endpoint in front of 100+ providers (OpenAI, Anthropic, Google, Azure, AWS Bedrock, Mistral, Groq, and more). Because the proxy speaks the OpenAI wire format, it needs no dedicated package — point `openaiCompatible` at your proxy's `baseURL` (default `http://localhost:4000/v1`) and route to a provider with LiteLLM's `provider/model` naming:

```typescript
import { openaiCompatible } from "@tanstack/ai-openai/compatible";

const litellm = openaiCompatible({
name: "litellm",
baseURL: "http://localhost:4000/v1", // your LiteLLM proxy
apiKey: process.env.LITELLM_API_KEY!, // a virtual key issued by the proxy
models: [
"anthropic/claude-sonnet-5",
"openai/gpt-5.5",
"gemini/gemini-3.5-flash",
],
});
```

`litellm("anthropic/claude-sonnet-5")` selects the Anthropic route; `litellm("openai/gpt-5.5")` selects OpenAI — all through the one proxy. Declare only the model routes you configured on the proxy; for precise per-model capabilities (e.g. a reasoning route without image input), use `createModel` as shown under [Declaring Models](#declaring-models).

> The proxy holds each upstream provider's real credentials; the `apiKey` here is the proxy's own virtual/master key, not the upstream provider's.

## Azure OpenAI

Azure uses a resource-scoped URL and a separate API-version. Use the `/openai/v1` endpoint with `defaultQuery` for the version and `defaultHeaders` for the `api-key` header:
Expand Down
2 changes: 1 addition & 1 deletion docs/config.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -586,7 +586,7 @@
"label": "OpenAI-Compatible",
"to": "adapters/openai-compatible",
"addedAt": "2026-06-01",
"updatedAt": "2026-06-20"
"updatedAt": "2026-07-20"
},
{
"label": "Claude Code",
Expand Down
Loading