LoMA

LoMA is a production-ready local AI ecosystem. It integrates Open WebUI with a custom LoMA-Agent via a dynamic pipeline system, providing an interface for local LLMs, agents, and session logging.

Structure

loma-openwebui/
├── loma_agent/ # Core Agent logic (FastAPI)
│ ├── agents.py # Agent definitions
│ ├── instructions.py # Agent instructions
│ ├── kb_loader.py # Knowledge base utilities
│ ├── logger.py # Database logging logic
│ ├── main.py # API Entry point
│ └── Dockerfile
├── ollama/ # Local LLM management
│ ├── Modelfile # Model configuration
│ ├── entrypoint.sh # Setup script
│ └── Dockerfile
├── eval/ # Agent evaluation
│ └── eval.py # Evaluation script
├── pipelines/ # OpenWebUI Pipeline Bridge
│ ├── pipelines_custom/ # OpenWebUI Pipelines here
│ │ └── loma_pipeline.py
│ ├── build.sh
│ └── Dockerfile
├── kb/ # Plain text knowledge base for FS navigation
├── models/ # Local .gguf files (Ollama)
├── data/ # RAG Dataset (dataset.jsonl)
├── .env # Environment variables
├── docker-compose.yml # Main services
├── docker-compose-gpu.yml # GPU Acceleration override
└── Makefile # Shortcuts for common tasks

Services Architecture

• open-webui: The frontend interface for chatting with the agent.
• pipelines: Automatically detects and attaches custom logic from pipelines_custom/ to OpenWebUI.
• agno-agent-api: A FastAPI server orchestration the Agent's reasoning and tools.
• ollama: Handles local LLM inference.
• qdrant: High-performance Vector Database for RAG.
• postgres-agno: Persistent storage for agent session logs and history.

Prerequisites

Clone the repository and prepare the following:

1. Environment: Create a .env file (see the Environment Variables section).
2. Models: Where .gguf files must be placed.
3. Knowledge Base:
   • Place raw documents in data/ as dataset.jsonl.
   • Generate the plain text KB for the agent using create_plain_kb method of kb_loader.py.

Installation and Execution

Using Makefile:

• GPU Mode:

   make gpu

• CPU Mode:

   make build

• Start without rebuilding:

   make up

Using Docker Compose directly:

• GPU Mode:

   docker compose -f docker-compose.yml -f docker-compose-gpu.yml up -d --build

• CPU Mode:

   docker compose up --build -d

Environment Variables

The .env file should contain at least the following environment variables:

# Postgres ConfigurationPOSTGRES_USR=your_userPOSTGRES_PWD=your_passwordPOSTGRES_PORT=5432# Ollama and ModelsOLLAMA_PORT=11434LOMA_MODEL=lomaEMBEDDING_MODEL=nomic-embed-text:latest# Qdrant and Agno External PortsAGNO_PORT=8000QDRANT_PORT=6333# PipelinesPIPELINES_PORT=9099PIPELINES_API_KEY=0p3n-w3bu!# OpenWebUI ConfigurationOPENWEBUI_PORT=3000

Notes

• Dynamic Pipelines: Any new script added to pipelines/pipelines_custom/ will be automatically loaded when the container starts.
• Persistence: Database logs are stored in the postgres-loma docker volume, ensuring history is kept across restarts.
• Fine-tuning: it possible to test fine-tuning using the code available on loma-fine-tuning