A privacy-first LLM proxy and AI gateway for teams that take control seriously.
VoidLLM is a self-hosted LLM proxy that sits between your applications and LLM providers - OpenAI, Anthropic, Azure, Ollama, vLLM, or any custom endpoint. It gives you organization-wide access control, API key management, usage tracking, rate limiting, and multi-deployment load balancing. One Go binary, sub-2ms proxy overhead, zero knowledge of your prompts.
More screenshots
Privacy-First by Design: VoidLLM is a zero-knowledge LLM proxy - it never stores, logs, or persists any prompt or response content. Not as a setting you can toggle - by architecture. Only metadata is tracked: who made the request, which model, how many tokens, how long it took. Your data stays yours.
| Problem | How VoidLLM solves it |
|---|---|
| Teams share raw API keys in Slack | Virtual keys with org/team/user scoping and RBAC |
| No visibility into who's spending what | Per-key, per-team, per-org usage tracking + cost estimation |
| One runaway script burns the monthly budget | Rate limits + token budgets enforced by the proxy at every level |
| Switching providers means changing every app | Model aliases - clients call default, the proxy routes it anywhere |
| Provider goes down, everything breaks | Multi-deployment load balancing with automatic failover |
| Existing proxies log your prompts | Zero-knowledge proxy architecture - content never touches disk |
flowchart LR
App[Your app / SDK] -->|OpenAI-compatible| Proxy[VoidLLM proxy]
Proxy --> Gate[API key + RBAC<br/>rate limits + budgets]
Gate --> Router[Model alias + load balancing]
Router --> Providers[OpenAI / Anthropic / Azure<br/>Ollama / vLLM / custom]
Proxy -. metadata only .-> DB[(SQLite / PostgreSQL)]
Your apps speak the OpenAI API to VoidLLM. It authenticates the key, applies RBAC, rate limits and budgets, resolves the model alias, and routes to the right provider. Only metadata - who called, which model, token counts, latency - is written to the database. Prompt and response content passes through memory and is never persisted.
A good fit if you:
- Self-host LLM infrastructure (vLLM, Ollama) or use managed providers and need one control plane
- Cannot log prompts or responses for privacy or compliance reasons
- Need org/team/user/key RBAC, budgets, and model routing
- Run multiple providers and want aliases, load balancing, and failover
Not the right fit if you:
- Want a hosted SaaS gateway - VoidLLM is self-hosted by design
- Need full prompt/response logging or content-level observability - it's zero-knowledge by architecture
- Need upstream MCP servers with per-user OAuth today - not yet supported
# Generate required keysexport VOIDLLM_ADMIN_KEY=$(openssl rand -base64 32)export VOIDLLM_ENCRYPTION_KEY=$(openssl rand -base64 32)# Start the proxy - no config file needed, VoidLLM boots with sensible defaults
docker run -p 8080:8080 \
-e VOIDLLM_ADMIN_KEY -e VOIDLLM_ENCRYPTION_KEY \
-v voidllm_data:/data \
ghcr.io/voidmind-io/voidllm:latestOn first start VoidLLM prints bootstrap credentials to stdout (shown below) - no config file is required. Add models in the UI, or mount a voidllm.yaml to declare them (see Configuration).
Download the latest binary for your platform from the releases page:
# Linux
curl -sL https://github.com/voidmind-io/voidllm/releases/latest/download/voidllm-linux-amd64.tar.gz | tar xz
export VOIDLLM_ADMIN_KEY=$(openssl rand -base64 32)export VOIDLLM_ENCRYPTION_KEY=$(openssl rand -base64 32)
./voidllmAvailable for: Linux (amd64, arm64), Windows (amd64, arm64), macOS (amd64, arm64).
On first start, VoidLLM prints your credentials to stdout:
========================================
BOOTSTRAP COMPLETE - COPY THESE NOW
========================================
API Key: vl_uk_a3f2...
Email: admin@voidllm.local
Password: <random>
========================================
Open http://localhost:8080, log in with the email and password above, and start proxying. The API key is used for SDK calls (Authorization: Bearer vl_uk_...). These credentials are shown once - save them.
Keys are auto-generated. Open the URL Railway gives you and start adding models.
# Your apps just point at the proxy instead of the provider
curl http://localhost:8080/v1/chat/completions \
-H "Authorization: Bearer vl_uk_..." \
-H "Content-Type: application/json" \
-d '{"model":"default","messages":[{"role":"user","content":"hello"}]}'Any OpenAI-compatible SDK works out of the box - just change the base URL to your VoidLLM proxy.
| Feature | Details |
|---|---|
| OpenAI-compatible proxy | /v1/chat/completions, embeddings, images, audio, streaming |
| Multi-provider routing | OpenAI, Anthropic, Azure, Ollama, vLLM, any custom endpoint |
| Load balancing | Round-robin, least-latency, weighted, priority across deployments |
| Automatic failover | Retry on 5xx/timeout, circuit breakers, health-aware routing |
| Web UI | Dashboard, playground, API keys, teams, models, usage, settings |
| RBAC | Org > Team > User > Key hierarchy, 4 roles |
| Rate limits | Requests per minute/day, most-restrictive-wins across levels |
| Token budgets | Daily/monthly limits, real-time enforcement |
| Usage tracking | Tokens, cost, duration, TTFT per request |
| Usage export | CSV / JSON download |
| Model aliases | Clients call default, you control where it routes |
| MCP gateway | Proxy external MCP servers with access control and session management |
| Code Mode | WASM-sandboxed JS for multi-tool orchestration |
| Prometheus metrics | Latency, tokens, active streams, routing, health |
| Database | SQLite (default) or PostgreSQL |
| Deployment | Docker, Helm chart, graceful shutdown |
| Pro (€49/mo · €490/yr) | Everything above, plus: |
| Unlimited orgs + teams | No limit on organizations or teams |
| Cost reports | Model breakdown, daily trends, budget alerts |
| Cross-org analytics | Usage and cost across all organizations |
| Support | Priority email (48h) |
| Enterprise (€149/mo · €1490/yr) | Everything in Pro, plus: |
| SSO / OIDC | Google, Azure AD, Okta, Keycloak, any provider |
| Per-org SSO | Each organization gets its own Identity Provider |
| Auto-provisioning | Users created from allowed email domains |
| Group sync | OIDC groups mapped to VoidLLM teams |
| Audit logs | Every admin action, filterable API + UI |
| OpenTelemetry | OTLP/gRPC export, request ID correlation |
| Multi-instance | Redis-backed rate limits and budgets across replicas |
| Support | Dedicated Slack (24h) |
Founding Member (€999 one-time): All Enterprise features (current and future), lifetime license, Product Advisory Board, direct founder access, priority support, early access. Limited spots.
Flat pricing - no per-user fees, no per-request charges. Self-hosted on your infrastructure.
VoidLLM is an MCP gateway - it exposes built-in management tools and proxies requests to external MCP servers with access control, usage tracking, and automatic session management.
| Tool | Description |
|---|---|
list_models | List models with health status (RBAC-scoped) |
get_model_health | Health status for a specific model or deployment |
get_usage | Usage stats for your key/team/org |
list_keys | API keys visible to you |
create_key | Create a temporary API key |
list_deployments | Deployment details (system_admin only) |
Register external MCP servers via the Admin UI or API. VoidLLM proxies tool calls through /api/v1/mcp/:alias with scoped access control (global, org, or team level), automatic session management, usage tracking, and Prometheus metrics.
Code Mode lets LLMs write JavaScript that orchestrates multiple MCP tool calls in a single execution - instead of one tool call per LLM turn. The JS runs in a WASM-sandboxed QuickJS runtime with no filesystem, no network, and no host access. Reduces token usage by 30-80%.
mcp:
code_mode:
enabled: truepool_size: 8# concurrent WASM runtimesmemory_limit_mb: 16# per executiontimeout: 30s# per executionmax_tool_calls: 50# per executionCode Mode exposes three tools on /api/v1/mcp:
| Tool | Description |
|---|---|
list_servers | Discover available MCP servers and tool counts |
search_tools | Find tools by keyword across all servers |
execute_code | Run JS with MCP tools as await tools.alias.toolName(args) |
TypeScript type declarations are auto-generated from tool schemas and included in the execute_code description, so LLMs see available tools and argument types at tools/list time.
Admins can block specific tools from Code Mode via the per-tool blocklist API and UI.
{
"mcpServers": {
"voidllm": {
"type": "http",
"url": "http://your-voidllm-instance:8080/api/v1/mcp",
"headers": { "Authorization": "Bearer vl_uk_your_key" }
}
}
}This connects your IDE (Claude Code, Cursor, Windsurf) to the Code Mode endpoint. Management tools (list_models, get_usage, etc.) are available at /api/v1/mcp/voidllm. External MCP servers at /api/v1/mcp/:alias.
- SSE transport not supported - MCP servers using the deprecated SSE protocol (pre 2025-03-26 spec) are auto-detected and deactivated. Use servers that support Streamable HTTP.
- No OAuth for upstream MCP servers - servers requiring per-user OAuth (Jira, Slack, Google) are not yet supported. API key and header auth work.
- Single instance only - Code Mode's WASM runtime pool is in-memory. Multi-pod deployments require Redis support (coming soon).
Full documentation | Blog | FAQ
| Topic | Guide |
|---|---|
| Getting Started | Quick Start |
| Configuration | All YAML settings |
| Docker | Docker deployment |
| Kubernetes | Helm chart |
| Providers | OpenAI, Anthropic, Azure, Ollama, vLLM |
| Load Balancing | Strategies, failover, circuit breakers |
| MCP Gateway | Overview - Servers - Code Mode - IDE Setup |
| RBAC | Roles and permissions |
| Privacy | Zero-knowledge architecture |
| API Reference | Endpoints and error codes |
| Enterprise | License - SSO - Audit - OTel - Pricing |
| Troubleshooting | Common issues |
server:
proxy:
port: 8080models:
# Single endpoint
- name: dolphin-mistralprovider: ollamabase_url: http://localhost:11434/v1timeout: 30saliases: [default]pricing:
input_per_1m: 0.15output_per_1m: 0.60# Load balanced - multiple deployments with failover
- name: gpt-4ostrategy: round-robinaliases: [smart]deployments:
- name: azure-eastprovider: azurebase_url: https://eastus.openai.azure.comapi_key: ${AZURE_EAST_KEY}azure_deployment: gpt-4opriority: 1
- name: openai-fallbackprovider: openaibase_url: https://api.openai.com/v1api_key: ${OPENAI_KEY}priority: 2mcp_servers:
- name: AWS Knowledgealias: awsurl: https://knowledge-mcp.global.api.awsauth_type: nonesettings:
admin_key: ${VOIDLLM_ADMIN_KEY}encryption_key: ${VOIDLLM_ENCRYPTION_KEY}mcp:
code_mode:
enabled: trueSupported providers: openai · anthropic · azure · vllm · ollama · custom
Environment variables are interpolated with ${VAR} syntax. Secrets never hardcoded.
cp voidllm.yaml.example voidllm.yaml
export VOIDLLM_ADMIN_KEY=$(openssl rand -base64 32)export VOIDLLM_ENCRYPTION_KEY=$(openssl rand -base64 32)
docker-compose uphelm install voidllm chart/voidllm/ \
--set secrets.adminKey=$(openssl rand -base64 32) \
--set secrets.encryptionKey=$(openssl rand -base64 32) \
--set config.models[0].name=my-model \
--set config.models[0].provider=ollama \
--set config.models[0].base_url=http://ollama:11434/v1PostgreSQL and Redis are available as optional subcharts for production deployments.
# Prerequisites: Go 1.23+, Node 20+cd ui && npm ci && npm run build &&cd ..
go run ./cmd/voidllm --config voidllm.yaml- Use PostgreSQL instead of SQLite
- Put VoidLLM behind TLS / a reverse proxy
- Isolate the admin UI/API from public proxy traffic (separate
admin.port, with TLS) - Use a strong
VOIDLLM_ENCRYPTION_KEY(32+ bytes) and keep it in a secrets manager - Don't use
VOIDLLM_ADMIN_KEYas a production API key - it's for bootstrap only - Set resource limits and network policies in Kubernetes
- Scrape
/metricswith Prometheus - Configure database backups
- For multiple replicas, use Redis so rate limits and budgets are shared (they are per-process without it) - Enterprise
See the Security Hardening guide for the full list.
This is not a feature toggle. It's an architectural decision that makes VoidLLM a privacy-first LLM proxy.
- No request body in logs, DB, or any persistent storage
- No response body in logs, DB, or any persistent storage
- No prompt caching - content passes through memory only
- Usage events contain only: who (key/org/team), what (model), how much (tokens/cost)
- There is no
enable_content_loggingoption. It doesn't exist. - Designed to support GDPR compliance - no personal data in prompts is stored or processed
# Bidirectional database migration
voidllm migrate --from sqlite:///data/voidllm.db --to postgres://user:pass@host/db
# License management (for Enterprise)
voidllm license verify < license.jwt- Contributing
- Changelog
- Security Policy - report vulnerabilities privately to security@voidmind.io
- Code of Conduct
Business Source License 1.1 - source available, self-hosting permitted, competing hosted services prohibited. Converts to Apache 2.0 four years after each release.
Built by VoidMind · voidllm.ai
This project was built with significant assistance from AI (Claude by Anthropic).