SQLite-AI

On-device LLM inference and embeddings inside SQLite.
Run Llama, Phi, Qwen, and embedding models directly from SQL queries — local, private, no API calls. GGUF support, GPU acceleration on Metal/CUDA.

Free managed instance → · Docs · Website · Blog

_{Data:Vector · Sync · Columnar · JS
AI:AI · Agent · Memory · MCP}

Want hybrid local + cloud AI? SQLite-AI runs models on-device; combine with SQLite Cloud to sync embeddings, share agent memory, or offload heavy inference. Free tier available.

---

SQLite-AI

SQLite-AI is an extension for SQLite that brings artificial intelligence capabilities directly into the database. It enables developers to run, fine-tune, and serve AI models from within SQLite using simple SQL queries — ideal for on-device and edge applications where low-latency and offline inference are critical. The extension is actively developed by SQLite AI, some API and features are still evolving.

Features

• Embedded AI Inference: Run transformer models directly from SQL queries.
• Streaming I/O: Token-by-token streaming via SQL aggregate functions.
• Fine-tuning & Embedding: On-device model customization and vector embedding.
• Full On-Device Support: Works on iOS, Android, Linux, macOS, and Windows.
• Offline-First: No server dependencies or internet connection required.
• Composable SQL Interface: AI + relational logic in a single unified layer.
• Audio Transcription: Speech-to-text via Whisper models (WAV, MP3, FLAC).
• Vision / Multimodal: Analyze images via multimodal models (JPG, PNG, BMP, GIF).
• Supports any GGUF model: available on Huggingface; Qwen, Gemma, Llama, DeepSeek and more

SQLite-AI supports text embedding generation for search and classification, a chat-like interface with history and token streaming, automatic context save and restore across sessions, audio transcription via Whisper models, and vision/multimodal image understanding — making it ideal for building conversational agents, memory-aware assistants, and voice-enabled applications.

Getting Started

# Start SQLite CLI
sqlite3 myapp.db

-- Load the extension
.load ./ai

Text Generation

-- Load a text generation modelSELECT llm_model_load('./models/Qwen2.5-3B-Q4_K_M.gguf', 'gpu_layers=99');
SELECT llm_context_create_textgen();
-- Generate textSELECT llm_text_generate('What is the most beautiful city in Italy?');

Embedding Generation

-- Load an embedding modelSELECT llm_model_load('./models/nomic-embed-text-v1.5-Q8_0.gguf', 'gpu_layers=99');
SELECT llm_context_create_embedding('embedding_type=FLOAT32');
-- Generate an embedding vectorSELECT llm_embed_generate('Hello world');
-- Generate an embedding as JSONSELECT llm_embed_generate('Hello world', 'json_output=1');

Chat

-- Load a chat modelSELECT llm_model_load('./models/Llama-3.2-3B-Instruct-Q4_K_M.gguf', 'gpu_layers=99');
SELECT llm_context_create_chat();
-- Send a message and get a complete responseSELECT llm_chat_respond('Tell me a joke.');
-- Or stream the reply token by tokenSELECT reply FROM llm_chat('Tell me another joke.');

Audio Transcription

-- Load a Whisper modelSELECT audio_model_load('./models/ggml-tiny.bin');
-- Transcribe from a file pathSELECT audio_model_transcribe('./audio/speech.wav');
-- Transcribe with optionsSELECT audio_model_transcribe('./audio/speech.mp3', 'language=it,translate=1');
-- Transcribe from a BLOB columnSELECT audio_model_transcribe(audio_data) FROM recordings WHERE id =1;

Vision / Multimodal

-- Load a multimodal model and its vision projectorSELECT llm_model_load('./models/Gemma-3-4B-IT-Q4_K_M.gguf', 'gpu_layers=99');
SELECT llm_context_create_textgen();
SELECT llm_vision_load('./models/mmproj-Gemma-3-4B-IT-f16.gguf');
-- Describe an imageSELECT llm_text_generate('Describe this image', './photos/cat.jpg');
-- Use vision in a chat conversationSELECT llm_context_create_chat();
SELECT llm_chat_respond('What do you see in this photo?', './photos/landscape.jpg');
-- Analyze multiple imagesSELECT llm_text_generate('Compare these two images', './img1.jpg', './img2.jpg');

Documentation

For detailed information on all available functions, their parameters, and examples, refer to the comprehensive API Reference.

Installation

Pre-built Binaries

Download the appropriate pre-built binary for your platform from the official Releases page:

• Linux: x86 and ARM
• macOS: x86 and ARM
• Windows: x86
• Android
• iOS

Loading the Extension

-- In SQLite CLI
.load ./ai
-- In SQLSELECT load_extension('./ai');

Swift Package

You can add this repository as a package dependency to your Swift project. After adding the package, you'll need to set up SQLite with extension loading by following steps 4 and 5 of this guide.

Here's an example of how to use the package:

import ai
...vardb:OpaquePointer?sqlite3_open(":memory:",&db)sqlite3_enable_load_extension(db,1)varerrMsg:UnsafeMutablePointer<Int8>?=nilsqlite3_load_extension(db, ai.path,nil,&errMsg)varstmt:OpaquePointer?sqlite3_prepare_v2(db,"SELECT ai_version()",-1,&stmt,nil)defer{sqlite3_finalize(stmt)}sqlite3_step(stmt)log("ai_version(): \(String(cString:sqlite3_column_text(stmt,0)))")sqlite3_close(db)

Android Package

Add the following to your Gradle dependencies:

implementation 'ai.sqlite:ai:0.7.55'

Here's an example of how to use the package:

SQLiteCustomExtensionaiExtension = newSQLiteCustomExtension(getApplicationInfo().nativeLibraryDir + "/ai", null);
SQLiteDatabaseConfigurationconfig = newSQLiteDatabaseConfiguration(
getCacheDir().getPath() + "/ai_test.db",
SQLiteDatabase.CREATE_IF_NECESSARY | SQLiteDatabase.OPEN_READWRITE,
Collections.emptyList(),
Collections.emptyList(),
Collections.singletonList(aiExtension)
);
SQLiteDatabasedb = SQLiteDatabase.openDatabase(config, null, null);

Note: Additional settings and configuration are required for a complete setup. For full implementation details, see the complete Android example.

Python Package

Python developers can quickly get started using the ready-to-use sqlite-ai package available on PyPI:

pip install sqlite-ai

For usage details and examples, see the Python package documentation.

Flutter Package

Add the sqlite_ai package to your project:

flutter pub add sqlite_ai # Flutter projects
dart pub add sqlite_ai # Dart projects

Usage with sqlite3 package:

import'package:sqlite3/sqlite3.dart';
import'package:sqlite_ai/sqlite_ai.dart';
sqlite3.loadSqliteAiExtension();
final db = sqlite3.openInMemory();
print(db.select('SELECT ai_version()'));

For a complete example, see the Flutter example.

---

License

This project is licensed under the Elastic License 2.0. You can use, copy, modify, and distribute it under the terms of the license for non-production use. For production or managed service use, please contact SQLite Cloud, Inc for a commercial license.

---

☁️ Hosted version

Need to coordinate AI across devices, fleets, or agents? SQLite Cloud is the managed backend for SQLite-AI — sync embeddings, share memory, and offload heavier inference when the device can't keep up.

Start free →

---

Part of the SQLite AI stack

SQLite-AI is one piece of a larger ecosystem that turns SQLite into a runtime for intelligent, distributed data:

Data layer

• sqlite-vector — ANN vector search inside SQLite
• sqlite-sync — Offline-first CRDT sync across devices
• sqlite-columnar — Column-oriented analytics for OLAP queries
• sqlite-js — Custom SQLite functions written in JavaScript

AI layer

• sqlite-ai — On-device LLM inference and embeddings (you are here)
• sqlite-agent — Autonomous AI agents running inside SQLite
• sqlite-memory — Persistent, searchable memory for agents
• sqlite-mcp — Call MCP tools directly from SQL queries

Managed platform

• SQLite Cloud — Hosted SQLite with sync, auth, edge functions, and analytics. Free tier →

Built by SQLite AI. Questions? Contact us.