llm-cli

The easiest LLM command-line tool. Remember credentials for multiple providers, switch between them instantly, and chat in your terminal.

Features

• 🔌 Ollama (native API) — no key needed, works out of the box
• 🤖 OpenAI-compatible — OpenAI, vLLM, LiteLLM, LocalAI, Groq, etc.
• 💾 Named presets — save your server/model configs, switch with one flag
• 💬 Interactive REPL — multi-turn conversation with history
• ⚡ Single-shot mode — pipe-friendly, great for scripts
• 🔑 API key storage — secure (~/.config/llm-cli/presets.json, mode 600)

Install

pip install -e .

Python 3.10+ required. Only dependency: httpx>=0.27.

Quick Start

# Single-shot (default preset is Ollama on localhost)
llm --prompt "How long do bees live?"# Specify preset and model explicitly
llm --preset ollama --model glm-5:cloud --prompt "Explain quantum entanglement in one sentence"# Use a system prompt
llm --system "You are a pirate. Respond only in pirate speak." --prompt "What is the weather today?"# Point directly at any OpenAI-compatible server
llm --url http://my-vllm:8000 --api-type openai --api-key mytoken --model meta-llama/Llama-3-8b-instruct --prompt "Hello"# Interactive shell (no --prompt)
llm
llm --preset ollama --model glm-5:cloud

Interactive Shell

$ llm
llm-cli v0.1.0 — interactive shell
Preset: ollama | Model: glm-5:cloud | URL: http://localhost:11434
Type a message to chat, or /help for commands.
Press Ctrl-C or type /exit to quit.
> How long do bees live for?
Worker bees live for about 6 weeks during summer...
> /preset list
NAME TYPE URL MODEL
─────────────────── ──────── ─────────────────────────────────── ────────────────────
ollama ollama http://localhost:11434 glm-5:cloud
> /preset add
── Add a new preset ──
Preset name: my-openai
Server URL: https://api.openai.com/v1
API type [openai]: openai
API key: sk-...
Default model: gpt-4o
Default system prompt: You are a helpful assistant.
✓ Preset 'my-openai' saved.
> /preset use my-openai
✓ Switched to preset 'my-openai' (model: gpt-4o)
> /model gpt-4o-mini
✓ Model set to 'gpt-4o-mini'
> /system You are an expert Python developer.
✓ System prompt set.
> /clear
✓ Conversation history cleared.
> /exit
Bye! 👋

Shell Commands

Command	Description
/help	Show all commands
/exit or /quit	Exit the shell
/status	Show current preset, model, system prompt, history length
/clear	Clear conversation history
/preset list	List all saved presets
/preset add	Add a new preset (interactive wizard)
/preset use NAME	Switch to a different preset
/preset remove NAME	Delete a preset
/model [NAME]	Show or set the current model
/system [TEXT]	Show or set the system prompt

CLI Flags

usage: llm [OPTIONS]
options:
-V, --version Show version
-h, --help Show help
connection:
-p, --preset NAME Preset name (default: 'ollama')
-u, --url URL Override server URL directly
--api-type TYPE API type with --url: 'ollama' or 'openai'
-k, --api-key KEY API key override
request:
-m, --model MODEL Model name override
-s, --system TEXT System prompt
--prompt TEXT One-shot prompt (omit for interactive shell)
--no-stream Disable streaming

Presets

Presets are stored in ~/.config/llm-cli/presets.json (permissions 600).

Default preset (auto-created on first run)

{
"ollama": {
"name": "ollama",
"api_url": "http://localhost:11434",
"api_type": "ollama",
"api_key": null,
"default_model": "glm-5:cloud",
"default_system_prompt": null
}
}

Example: multiple presets

{
"ollama": { ... },
"openai": {
"name": "openai",
"api_url": "https://api.openai.com/v1",
"api_type": "openai",
"api_key": "sk-...",
"default_model": "gpt-4o"
},
"my-vllm": {
"name": "my-vllm",
"api_url": "http://my-server:8000",
"api_type": "openai",
"api_key": "token-xyz",
"default_model": "meta-llama/Llama-3-8B"
}
}

Supported Servers

Server	api_type	Notes
Ollama	ollama	Uses /api/chat NDJSON streaming
OpenAI	openai	Requires API key
vLLM	openai	OpenAI-compatible
LiteLLM	openai	OpenAI-compatible
LocalAI	openai	OpenAI-compatible
Groq	openai	Base URL: https://api.groq.com/openai/v1
Together AI	openai	OpenAI-compatible

License

MIT