Atomic Local Ai

Website · Docs · Discord · X

Open-source local AI — agents, chat, and inference. Private by default.

An open-source desktop AI app. Run local LLMs from Hugging Face or connect cloud models (OpenAI, Anthropic, Mistral, Groq, MiniMax, others). Available on macOS, Windows, Linux, iOS, and Android.

Desktop

Mobile

curl http://localhost:1337/v1/chat/completions -d '{ "model": "llama-3.2-3b-instruct", "messages": [{ "role": "user", "content": "Why is the sky blue?" }]}'

• Run open-weight LLMs (Llama, Gemma, Qwen, Mistral, Phi) from Hugging Face — fully offline
• Connect cloud providers: OpenAI, Anthropic, Mistral, Groq, MiniMax
• One-click agent launch — OpenCode & GitHub Copilot CLI from the Integrations tab
• Artifacts — live preview panel for HTML/CSS/JS
• Faster on Apple Silicon — MLX EAGLE-3, MTP, and TurboQuant KV cache
• MCP integration for agentic capabilities; custom assistants for specialized tasks
• Native iOS & Android apps, not wrappers

A local-first AI agent that runs on your machine, with local or cloud models. It drives your browser, reads and edits files, runs approved shell commands, and remembers context across sessions. Runs on our TurboQuant llama.cpp, so small quantized models stay useful for long, multi-step work on ordinary hardware.

macOS / Linux:

curl -fsSL https://api.atomicbot.ai/agent-install | sh

Windows (PowerShell):

irm https://raw.githubusercontent.com/AtomicBot-ai/atomic-agent/main/scripts/install.ps1 | iex

atomic-agent

• System browser via ARIA snapshots, shell, filesystem, documents (PDF/DOCX/XLSX), git, clipboard, HTTP, notifications
• GBNF grammar-constrained tool calls, parallel tool batches, cache-hot prompt prefix, externalized state in SQLite
• Local Markdown skills loaded on demand, FTS5 note recall, durable cron and webhook-triggered tasks
• TUI, CLI, OpenAI-compatible HTTP server, and a Tauri sidecar speaking newline-delimited JSON
• Policy-gated dangerous actions, append-only NDJSON traces with prompt-drift replay

A llama.cpp fork with TurboQuant KV cache compression and Gemma 4 MTP speculative decoding. ~30-50% throughput gains on the same hardware, drop-in compatible with upstream tools and GGUF.

WHT-rotated 2/3/4-bit KV cache with backend-native kernels (Metal TurboFlash, CUDA, Vulkan, HIP). turbo3 is the default — 3-bit, ~4.3× compression vs F16.

llama-server -m model.gguf -c 32768 -ngl 99 -fa on \
-ctk turbo3 -ctv turbo3

Pair any gemma4 target with the official gemma4_assistant head — loaded into the target context, no second tokenizer or KV cache. +30-50% short-prompt throughput on Gemma 4 26B-A4B / 31B at 85-88% accept rate. Pre-built assistant heads on Hugging Face.

llama-server -m gemma-4-target.gguf -c 16384 -ngl 99 -ngld 99 -fa on \
--mtp-head gemma-4-assistant-Q4_K_M.gguf \
--spec-type mtp --draft-block-size 3

• TQ3_1S / TQ4_1S weight quantization via llama-quantize — 25-35% smaller than Q8_0, single-digit % PPL delta
• Regularly synced with ggml-org/llama.cpp
• Powers local inference in Atomic Chat and Atomic Agent

_{© 2026 Atomic · atomicbot.ai}