A provider-scoped LLM proxy with OpenAI Chat Completions and Anthropic Messages client endpoints, normalized protocol translation, streaming, model catalogs, and optional live model discovery.
apps/llm-proxy main binary
crates/llm-proxy-core config, types, traits
crates/llm-proxy-protocol OpenAI / Claude / Gemini wire types
crates/llm-proxy-provider upstream channel implementations
crates/llm-proxy-storage persistence
crates/llm-proxy-api admin and user HTTP API
crates/llm-proxy-server axum wiring
cargo build --release
./target/release/llm-proxy validate --config ./config.toml
./target/release/llm-proxy serve --config ./config.toml
Every API request names its provider in the path:
POST /providers/{provider}/v1/chat/completions
POST /providers/{provider}/v1/messages
POST /providers/{provider}/v1/messages/count_tokens
GET /providers/{provider}/v1/models
See config.toml.example and the files under providers/ for configuration
examples.
See the protocol design and research index for architecture and compatibility notes.
These shaped the layout and scope:
- LeenHawk/gproxy
- modpotatodotdev/LLMG
- x5iu/openproxy
- habibi-dev/rust-llm-proxy