locat — a local, fully-offline Pipecat voice bot

WIP but isn't everything?

A voice bot that runs Key-Free & 100% offline. Speech-to-text, the language model, and text-to-speech are all local services, and the "transport†" is your machine's own audio hardware — the microphone and speakers.

Built on the latest Pipecat release (≥ 1.7). This repo is meant to double as a clear, reproducible example of how to wire up a fully-local Pipecat bot.

The v1 personality is a private financial thinking partner: something you can talk through money decisions with, out loud, knowing nothing you say leaves the computer.

†Audio is hard and there are a few ways to handle it in this scenario. See "how do you solve a problem like echo cancellation?"

Requirements

Apple Silicon Mac recommended — Whisper-MLX (the default STT) uses Apple's MLX framework and only runs there. Intel Macs and Linux work too: they default to CPU STT (faster_whisper) automatically. Windows: not yet (WSL works).
~15 GB free disk for the models
Python 3.12 is pinned as 3.14 is too new for the ML wheels.
A plain uv sync needs no compiler — everything in the base install comes from a prebuilt wheel. Several deps (onnxruntime, numba/llvmlite, cryptography) have already dropped Intel-mac wheels, so pyproject.toml pins those back to their last Intel-mac release under [tool.uv]. If uv sync ever starts building something from source, run python3 scripts/check_wheels.py to see which platform lost a wheel. The one exception is PyAudio, which has no macOS/Linux wheels — so it is an opt-in extra (--extra local-audio) needed only by the headphones front-end.
uv — Python package manager.
Ollama — serves the local LLM.
PortAudio OR any web browser — PyAudio's native dependency / audio handling. Browser front-ends need neither PortAudio nor a compiler.

Quickstart Setup

0. The short short version:

Note

The first pull will take a few minutes to download the models.

Browser-based (for echo cancellation):

git clone git@github.com:vipyne/locat.git &&cd locat
uv sync
bash scripts/run_ollama.sh
uv run python scripts/prefetch_models.py

Ctrl+C; then turn off wi-fi if you want to show off and then:

./start.sh

Open http://localhost:7860, choose "Media over QUIC", click Connect & have a conversation. (Prefer WebRTC? ./start.sh -t webrtc → http://localhost:7860/client.)

Or...

PyAudio & headphones (for echo cancellation):

Important

Use headphones 🎧

git clone git@github.com:vipyne/locat.git &&cd locat
brew install portaudio # Debian: sudo apt install portaudio19-dev
uv sync --extra local-audio # the extra adds PyAudio (needs PortAudio)
bash scripts/run_ollama.sh
uv run python scripts/prefetch_models.py

Ctrl+C; then turn off wi-fi if you want to show off and then:

./start.sh -t headphones

Have a conversation.

Setup

1. Clone, install system deps, and sync the environment

git clone git@github.com:vipyne/locat.git &&cd locat
uv sync # creates .venv and installs everything (Python 3.12)

That is all you need for the browser front-ends. For the headphones front-end (bot.py), PyAudio has to compile against PortAudio, so install it and opt into the extra:

brew install portaudio # Debian: sudo apt install portaudio19-dev
uv sync --extra local-audio

Note

uv sync uninstalls any extra you don't pass, so keep listing the ones you want: uv sync --extra local-audio --extra piper. (./doctor.sh -i preserves whatever is already installed.)

Optionally copy the config template (everything is optional — the bot runs with an empty or absent .env):

cp env.example .env

2. Fetch the models (the one-time online step)

Four model-backed components need weights. Two download from Hugging Face (anonymously — none are gated); the LLM is pulled by Ollama. Silero VAD and Smart Turn v3 ship inside the Pipecat package, so they download nothing.

All checkpoints are steered into the repo-local ./models/ tree (gitignored), so everything the bot needs lives next to the code.

a) Pull the LLM into the repo's Ollama store:

bash scripts/run_ollama.sh

This relocates Ollama's model store to ./models/ollama, starts ollama serve, pulls the model (qwen2.5:14b by default, ~9 GB), and keeps the server running in the foreground for the bot. Override the model with LLM_MODEL=qwen2.5:7b bash scripts/run_ollama.sh. Leave this running (or re-run it) whenever you use the bot — it's the local LLM server.

b) Prefetch the Whisper + Kokoro weights:

uv run python scripts/prefetch_models.py

Downloads Whisper-MLX (large-v3-turbo, ~1.5 GB) into ./models/huggingface and Kokoro's ONNX model + voices (~350 MB) into ./models/kokoro, and load-checks the bundled Silero VAD + Smart Turn v3 (no download). Run this once, while online; after it finishes the bot can run with Wi-Fi off.

Approximate total download: ~11 GB (9 GB LLM + 1.5 GB Whisper + 0.35 GB Kokoro).

3. Run

Important

Use headphones 🎧

With the Ollama server from step 2a running:

uv run bot.py

The bot speaks a short greeting, then listens. Talk to it; it replies through your speakers. Talk over it and it yields (barge-in). Press Ctrl-C to stop.

One-command launch:./start.sh brings up the repo-local Ollama server (if it isn't already running), prints the exact STT/LLM/TTS models in play, and serves the MoQ browser bot — so you can skip the manual run_ollama.sh in step 2a. Pick a different transport with -t: ./start.sh -t webrtc (browser, SmallWebRTC) or ./start.sh -t headphones (local audio hardware) — see echo cancellation. Not sure what your machine can handle? ./doctor.sh prints recommended STT/LLM/TTS cascades sized to your hardware (add -v for the full hardware profile and per-slot model catalogs ranked by fit, or -i to interactively pick a combo the script sanity-checks against your hardware).

4. Run offline

Once the models are fetched:

Make sure the local Ollama server is running (bash scripts/run_ollama.sh).
Turn off Wi-Fi / enable Airplane Mode. (It won't use the internet if you don't turn off the internet. This is just showing off.)
uv run bot.py and hold a conversation.

With LOG_LEVEL=DEBUG (the default) you can watch the logs and confirm no service reaches out to the network after the warm-up.

How do you solve a problem like echo cancellation

Use headphones

Because reasons, it's much closer to impossible than just impractical to get native macOS AEC (Acoustic Echo Cancellation) to work with pyaudio. Use headphones and the bot won't keep interrupting itself.

Use the web browser's `getUserMedia`

Another fantastic workaround is to use a browser. Not the internet, just the web browser. Do this and 🎉, you have echo cancellation.

Two browser transports ship here — same offline brain, different transport. start.sh brings up Ollama and serves a local page (still fully offline — the browser talks to the bot over loopback, no internet):

./start.sh # MoQ/QUIC → open http://localhost:7860, pick "Media over QUIC"
./start.sh -t webrtc # WebRTC → open http://localhost:7860/client

No secrets, no keys

There are no API keys anywhere in this project, and there's nowhere to put one:

Ollama pulls the LLM from its own public registry and serves it locally.
Whisper-MLX, Kokoro, Silero VAD, Smart Turn v3 download anonymously from Hugging Face (none are gated) — or, for Silero/Smart Turn, ship bundled with Pipecat.

.env is config only — model names, a voice, device indices, cache paths. It is gitignored, but nothing secret ever belongs in it. The single network event in the bot's entire lifecycle is the one-time, anonymous model download in step 2.

What's inside

Component	Service	Notes
Speech-to-text	`WhisperSTTServiceMLX`(default)	Apple-Silicon-optimized Whisper via MLX. Alternatives via `STT_ENGINE`: `faster_whisper` (CPU), `moonshine` (tiny CPU ONNX)
Language model	Qwen2.5-14B-Instruct via Ollama	Local, OpenAI-compatible endpoint; env-configurable
Text-to-speech	`KokoroTTSService`(default)	Natural local neural voice (kokoro-onnx). Alternative via `TTS_ENGINE`: `piper`
Turn-taking	Silero VAD + Local Smart Turn v3	Barge-in / interruptions, fully local (bundled with Pipecat)
Transport	`LocalAudioTransport`	PyAudio mic + speaker I/O (requires headphones)
Alternative transports	`SmallWebRTC` / `MoQ`	run in a browser → free echo cancellation via `getUserMedia`

Configuration

Every knob is an environment variable (read from .env if present). All are optional — the shown value is the default. See env.example for the copy-paste template.

Variable	Default	What it does
`LLM_MODEL`	`qwen2.5:14b`	Ollama model tag. Same string `run_ollama.sh` pulls and the bot serves. Smaller/faster: `qwen2.5:7b`.
`OLLAMA_BASE_URL`	`http://localhost:11434/v1`	OpenAI-compatible Ollama endpoint (note the trailing `/v1`).
`STT_ENGINE`	`whisper_mlx`*	STT engine `services.py` builds: `whisper_mlx`, `faster_whisper`, or `moonshine` (`uv sync --extra moonshine`). *Default is `faster_whisper` on non-Apple-Silicon machines.
`WHISPER_MODEL`	`LARGE_V3_TURBO`	`MLXModel` member: `TINY`, `MEDIUM`, `LARGE_V3`, `LARGE_V3_TURBO`. Must match what you prefetched.
`FASTER_WHISPER_MODEL`	`DISTIL_MEDIUM_EN`	faster-whisper model (when `STT_ENGINE=faster_whisper`); downloads on first use.
`MOONSHINE_MODEL`	`SMALL_STREAMING`	Moonshine model (when `STT_ENGINE=moonshine`); downloads on first use.
`TTS_ENGINE`	`kokoro`	TTS engine `services.py` builds: `kokoro` or `piper` (`uv sync --extra piper`; piper-tts is GPL-3.0).
`KOKORO_VOICE`	`af_heart`	Kokoro voice id (e.g. `af_bella`, `am_michael`, `bf_emma`).
`PIPER_VOICE`	`en_US-lessac-medium`	Piper voice id (when `TTS_ENGINE=piper`); downloads (~60 MB) on first use into `./models/piper`.
`INPUT_DEVICE_INDEX`	(system default)	PyAudio mic index.
`OUTPUT_DEVICE_INDEX`	(system default)	PyAudio speaker index.
`GREETING`	"Hi. I'm your private, offline financial thinking partner…"	Opening line spoken on startup.
`GREETING_DELAY_SECS`	`1.0`	Delay before the greeting (lets the audio-out stream spin up).
`LOG_LEVEL`	`DEBUG`	Loguru level for stderr. `DEBUG` surfaces each service's activity — handy for the offline check.
`HF_HOME`	`./models/huggingface`	Hugging Face cache root (Whisper-MLX weights). Advanced.
`KOKORO_MODEL_PATH`	`./models/kokoro/kokoro-v1.0.onnx`	Kokoro ONNX model path. Advanced.
`KOKORO_VOICES_PATH`	`./models/kokoro/voices-v1.0.bin`	Kokoro voices bundle path. Advanced.
`OLLAMA_MODELS`	`./models/ollama`	Ollama store location (used by `run_ollama.sh`). Advanced.
`OLLAMA_HOST`	`127.0.0.1:11434`	Host the Ollama server binds to (used by `run_ollama.sh`). Advanced.

Changing LLM_MODEL swaps which local model answers; changing KOKORO_VOICE changes the voice you hear.

Repository layout

All three bots share one offline brain (the same STT → VAD → LLM → TTS pipeline); they differ only in the transport. The STT/LLM/TTS services themselves are built in services.py, dispatched on STT_ENGINE / TTS_ENGINE — so swapping engines (via .env or ./doctor.sh -i) never touches a bot file you may have customized.

locat/
├── bot.py # CLI / headphones — LocalAudioTransport
├── bot_web.py # browser / speakers — SmallWebRTC (free echo cancellation)
├── bot_moq.py # browser / speakers — MoQ over QUIC (lower latency)
├── services.py # STT/LLM/TTS builders, engine-dispatched (STT_ENGINE / TTS_ENGINE)
├── config.py # env-driven settings, zero-config defaults
├── spoken_text_filter.py # TTS filter: "$3,000" → "three thousand dollars"
├── prompts/
│ └── financial_advisor.py # the v1 system prompt
│
├── start.sh # one command: bring up Ollama + run the bot (-t moq|webrtc|headphones)
├── doctor.sh # what can this machine handle? (-v full report, -i model picker)
├── stop.sh # stop the background Ollama server
│
├── scripts/
│ ├── run_ollama.sh # relocate Ollama store + serve + pull the LLM
│ ├── prefetch_models.py # one-time online warm-up (Whisper + Kokoro)
│ ├── check_audio.py # diagnostic: raw mic input level meter
│ └── check_vad.py # diagnostic: Silero VAD confidence/volume vs thresholds
│
├── ralph/ # the "ralph loop" that built this repo
│ ├── ralph.sh # autonomous agent runner
│ ├── PROMPT.md # per-iteration instructions for the loop
│ ├── RALPH.md # operator runbook for the loop
│ └── PLAN.md # the approved build plan the loop followed
│
├── MODELS_TO_ADD.md # engines considered but not (yet) wired — and why
├── env.example # documented config knobs (copy to .env)
├── .python-version # 3.12
├── pyproject.toml # uv project + pinned deps
├── uv.lock # locked dependency versions
│
└── models/ # ALL checkpoints live here (gitignored; created by setup)
├── huggingface/ # Whisper-MLX + faster-whisper (HF cache)
├── kokoro/ # Kokoro onnx + voices
├── piper/ # Piper voices (if TTS_ENGINE=piper)
└── ollama/ # Ollama LLM store

Roadmap

v1 is conversation only; the repo is structured so later capabilities layer in cleanly, each its own build cycle:

create a fun custom frontend for the browser versions.
Document RAG over your own financial files (local embeddings + vector store).
Function-calling tools (compound interest, amortization, savings-goal calculators).
Persistent memory across sessions (local JSON/SQLite).

Not financial advice

The bot is a private thinking partner, not a licensed financial advisor. It has no access to your real accounts and won't invent your numbers. For big, irreversible, or high-stakes decisions, confirm with a qualified professional. Ha, claude wrote this^ when I said I wanted to create a fully offline bot that I could talk to about my personal finances. But yes, always consult a human after consulting a bot.

Emojis

claude did not add enough/any emojis so: 🎉🎊🥳🎈🎁🎀🌟✨💫⭐🌈🔥💥⚡☀️🌙🌛🌜🌞🪐🌍🌎🌏🌊🏔️⛰️🌋🗻🏕️🏖️🏜️🏝️🌅🌄🌇🌆🏙️🌃🌌🎆🎇🌠🌉🍀🌿🍃🌾🌵🌴🌳🌲🎄🌰🍄🌻🌺🌸🌼🌷🌹🥀💐🏵️🌊🐠🐟🐬🐳🐋🦈🐙🦑🦐🦞🦀🐚🐌🦋🐛🐝🐞🦗🕷️🦂🐢🐍🦎🦖🦕🐙🦭🦦🦥🐾🐕🐈🐇🐿️ 🦫🦃🐔🐓🐣🐤🐥🦆🦢🦅🦉🦚🦜🕊️🐧🐦🦩🦨🐘🦏🦛🐪🐫🦒🦓🐂🐃🐄🐎🐖🐏🐑🦙🐐🦌🐕‍🦺🐈‍⬛🦮🐩🐾🍎🍏🍐🍊🍋🍌🫐🍈🍒🍑🥭🍍🥥🥝🍅🍆🥑🥦🥬🥒🌶️ 🫑🌽🥕🫒🧄🧅🥔🍠🥐🥯🍞🥖🥨🧀🥚🍳🧈🥞🧇🥓🥩🍗🍖🌭🍔🍟🍕🫓🥪🥙🧆🌮🌯🫔🥗🥘🫕🥫🍝🍜🍲🍛🍣🍱🥟🦪🍤🍙🍚🍘🍥🥠🥮🍢🍡🍧🍨🍦🥧🧁🍰🎂🍮🍭🍬🍫🍿🍩🍪🌰🥜🍯🥛🍼☕🫖🍵🧃🥤🧋🍶🍺🍻🥂🍷🥃🍸🍹🧉🍾🧊🥄🍴🍽️🥣🥡🥢🧂⚽🏀🏈⚾🥎🎾🏐🏉🥏🎱🪀🏓🏸🏒🏑🥍🏏🪃🥅⛳🪁🏹🎣🤿🥊🥋🎽🛹🛼🛷⛸️ 🥌🎿⛷️ 🏂🪂🏋️ 🤼🤸⛹️ 🤺🤾🏌️ 🏇🧘🏄🏊🤽🚣🧗🚵🚴🏆🥇🥈🥉🏅🎖️ 🏵️ 🎗️ 🎫🎟️ 🎪🤹🎭🩰🎨🎬🎤🎧🎼🎹🥁🎷🎺🎸🪕🎻🎲♟️🎯🎳🎮🎰🧩

Navigation Menu

Repository files navigation

locat — a local, fully-offline Pipecat voice bot

Requirements

Quickstart Setup

0. The short short version:

Browser-based (for echo cancellation):

PyAudio & headphones (for echo cancellation):

Setup

1. Clone, install system deps, and sync the environment

2. Fetch the models (the one-time online step)

3. Run

4. Run offline

How do you solve a problem like echo cancellation

Use headphones

Use the web browser's `getUserMedia`

No secrets, no keys

What's inside

Configuration

Repository layout

Roadmap

Not financial advice

Emojis

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
prompts		prompts
ralph		ralph
scripts		scripts
.gitignore		.gitignore
.python-version		.python-version
MODELS_TO_ADD.md		MODELS_TO_ADD.md
README.md		README.md
bot.py		bot.py
bot_moq.py		bot_moq.py
bot_web.py		bot_web.py
config.py		config.py
doctor.sh		doctor.sh
env.example		env.example
pyproject.toml		pyproject.toml
services.py		services.py
spoken_text_filter.py		spoken_text_filter.py
start.sh		start.sh
stop.sh		stop.sh
uv.lock		uv.lock

Latest commit

History

Folders and files

Repository files navigation

locat — a local, fully-offline Pipecat voice bot

Requirements

Quickstart Setup

0. The short short version:

Browser-based (for echo cancellation):

PyAudio & headphones (for echo cancellation):

Setup

1. Clone, install system deps, and sync the environment

2. Fetch the models (the one-time online step)

3. Run

4. Run offline

How do you solve a problem like echo cancellation

Use headphones

Use the web browser's getUserMedia

No secrets, no keys

What's inside

Configuration

Repository layout

Roadmap

Not financial advice

Emojis

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Use the web browser's `getUserMedia`