LocalRAG

LocalRAG is a lightweight, zero-dependency C++17 Retrieval-Augmented Generation (RAG) assistant designed for local workspaces. It builds a search index of local text and Markdown files using a custom Binary Search Tree (BST) multimap, retrieves relevant context for user queries, and generates grounded, factual answers using Large Language Models via the OpenRouter API.

LocalRAG is designed with high-performance modern C++ idioms, robust HTTP error handling, and search engine optimizations suitable for real-world document Q&A.

---

Key Features

• 🚀 High-Performance BST Indexing: Utilizes a generic, fully iterative BSTMultimap implementation. Node insertions and searches use loops (no recursion) and binary search insertion (std::lower_bound) to keep token lists sorted.
• ⚡ Zero-Copy Tokenization: Processes document text using std::string_view and standard library search algorithms (std::find_if), avoiding unnecessary memory allocations.
• 🔍 Optimized Multi-Term Queries: Intersects document lists in linear time $O(N + M)$ using std::set_intersection. Queries are optimized by sorting document lists by size and intersecting the smallest lists first.
• 🌐 Robust LLM Connector: Calls the OpenRouter API (using qwen/qwen-2.5-7b-instruct by default) via libcurl. Employs JSON escaping and structured response extraction.
• 🔐 Secure Key Management: Supports environment variables (OPENROUTER_API_KEY) and .env configuration files to keep credentials secure.
• 🎨 ANSI Styled CLI: Features a terminal interface with ANSI color coding, robust CLI argument parsing, and clear help commands.

---

Repository Architecture

The project is structured logically separating interface and implementation:

• include/bst_multimap.h: The generic template-based binary search tree multimap.
• include/tokenizer.h / src/tokenizer.cpp: Zero-copy string lexer.
• include/knowledge_index.h / src/knowledge_index.cpp: Builds the search index and performs optimized intersections.
• include/llm_client.h / src/llm_client.cpp: libcurl communicator for OpenRouter APIs.
• include/query_engine.h / src/query_engine.cpp: Coordinates the RAG prompt orchestration pipeline.
• src/main.cpp: The CLI application entry point.

---

Prerequisites

Before building LocalRAG, ensure you have the following installed:

• A C++17 compatible compiler (e.g., g++ or clang++).
• libcurl developer libraries.
  • macOS: Pre-installed.
  • Ubuntu/Debian: sudo apt install libcurl4-openssl-dev

---

Setup & Compilation

1. Build the Executable

Build the project using the provided Makefile:

make

This compiles the source files and outputs a localrag executable in the root directory.

2. Configure your API Key

Create a .env file in the root directory by copying the example:

cp .env.example .env

Open .env and add your OpenRouter API key:

OPENROUTER_API_KEY=your_api_key_here

Alternatively, you can export the key directly to your environment:

export OPENROUTER_API_KEY="your_api_key_here"

---

How to Run

Provide the directory containing your knowledge base (Markdown or text files) as an argument:

./localrag notes

Options

To view help and CLI options, run:

./localrag --help

---

License

This project is licensed under the MIT License - see the LICENSE file for details.