5th Member — Local RAG AI Chat Engine

5th Member is a self-hosted conversational AI stack that remembers over time using Ollama, Qdrant, and FastAPI.
It acts as your team’s quiet 5th member — listening, learning, and responding with context-aware intelligence.

---

Features

• Conversational Memory: Stores and recalls chat history through Qdrant vector search
• RAG Integration: Retrieves past context to enrich current prompts automatically
• FastAPI Backend: Lightweight async API server ready for local or containerized deployment
• Ollama Integration: Uses locally-running Ollama models for offline or private inference
• Docker Support: Works seamlessly with local Dockerized Qdrant setup
• Progressive Summarization: Keeps your long-term memory concise and relevant

---

Tech Stack

Component	Purpose
FastAPI	API framework for async chat & RAG endpoints
Ollama	Local LLM runner (e.g., llama3, mistral, codellama)
Qdrant	Vector database to store and retrieve conversation embeddings
httpx	Async HTTP client for streaming LLM responses
Python 3.11+	Core runtime

---

Setup Guide

1️⃣ Clone the Repository

git clone https://github.com/utsav-develops/5thMember.git
cd 5thMember

2️⃣ Create & Activate Virtual Environment

python -m venv .venv
# Activate:source .venv/bin/activate # macOS / Linux# or
.venv\Scripts\activate # Windows

3️⃣ Install Dependencies

pip install -r requirements.txt

---

🧩 Environment Configuration

Create a .env file in your project root directory:

# Ollama SettingsOLLAMA_URL=http://localhost:11434OLLAMA_TEXT_MODEL=llama3OLLAMA_EMBED_MODEL=embedding-model# Qdrant SettingsQDRANT_URL=http://localhost:6333QDRANT_COLLECTION=memories

---

🐳 Running Services

🧠 Run Qdrant via Docker

docker run -p 6333:6333 -v qdrant_storage:/qdrant/storage qdrant/qdrant

⚙️ Run Ollama (Local LLM)

ollama pull llama3
ollama serve

💡 You can also use other models like mistral, codellama, or phi3 by updating the .env file.

---

🚀 Launch the FastAPI Server

uvicorn main:app --reload --port 8000

Server runs at: http://localhost:8000

---

💡 Usage Examples

🔹 Basic Chat Endpoint

Send messages via /chat:

curl -X POST http://localhost:8000/chat \
-H "Content-Type: application/json" \
-d '{"prompt": "Explain quantum entanglement"}'

🔹 RAG Chat Endpoint (with memory)

Uses Qdrant-stored context to enhance replies:

curl -X POST http://localhost:8000/rag-chat \
-H "Content-Type: application/json" \
-d '{"user_id": "user123", "prompt": "Remind me what we discussed earlier about ML agents"}'

---

🧠 How RAG Memory Works

Every message is embedded and stored in Qdrant. When a new message arrives:

1. Qdrant searches for similar past messages by vector similarity
2. Relevant context is retrieved
3. A new, context-rich prompt is constructed
4. Ollama generates a response with awareness of previous conversations
5. Older memories are periodically summarized and compressed

---

📂 Project Structure

5thMember/
│
├── main.py # FastAPI app and API endpoints
├── utils.py # RAG logic, summarization, Qdrant operations
├── db.py # Database (Qdrant) user management
├── requirements.txt # Python dependencies
├── .env # Environment variables
└── README.md # Documentation (you’re reading this!)

---

🧰 Example Models

You can use any Ollama-supported model:

ollama pull mistral
ollama pull phi3
ollama pull codellama

Then, update .env:

OLLAMA_TEXT_MODEL=phi3

---

🧑‍💻 Development Tips

• Use --reload flag in Uvicorn for hot-reloading during development

• Check your Qdrant dashboard at http://localhost:6333/dashboard

• Logs can be viewed via Docker:

  docker logs <container_id>

---

🧾 Example .env Template

# LLM SettingsOLLAMA_URL=http://localhost:11434OLLAMA_TEXT_MODEL=llama3OLLAMA_EMBED_MODEL=embedding-model# Vector DBQDRANT_URL=http://localhost:6333QDRANT_COLLECTION=memories

---

📜 License

MIT License © 2025 — Created by Utsav Acharya

“The best teammate never sleeps — it just keeps learning.”