support-platform

Multi-tenant knowledge and support platform. Companies upload their documents and connect their customer database; end users ask questions in natural language; an agent decides between retrieval-augmented generation, text-to-SQL, and ticket creation, calls tools in parallel, and answers with citations over a streaming HTTP API.

Features

• Hybrid retrieval: dense embeddings (OpenAI text-embedding-3-small) + sparse BM25 (Qdrant, IDF-modified) fused via Reciprocal Rank Fusion, then reranked by Cohere rerank-v3.5 for top-K precision
• Agent loop with parallel tool calls: ReAct-style iteration with search_docs, query_database, create_ticket, and task_complete; tools dispatched concurrently when the model selects more than one in a single turn
• Text-to-SQL with AST safety: LLM-generated SQL validated via sqlglot — SELECT-only, table whitelist, mandatory workspace_id literal filter, forbidden operations rejected at the AST node level
• SSE streaming chat: real-time tool_start, tool_result, and done events with the final payload carrying answer, citations, iteration count, and USD cost
• Cost tracking: per-call token and dollar accounting for input, output, cache reads, and cache writes; aggregated by workspace and model through an admin endpoint
• Conversation memory: sliding window of recent turns plus LLM-driven summarization for context that grows beyond the window
• Multi-tenant isolation: workspace_id enforced at every layer — Postgres repository predicates, Qdrant payload filters, tool constructor binding, SQL validator literal check
• Production hardening: prompt-injection defenses (XML user-message wrap, untrusted tool-output convention, system prompt leakage refusal), output truncation, structured error mapping (RFC 7807), structured logging with correlation IDs propagated through async task chains

Tech stack

Layer	Choice
Language	Python 3.11+
Package manager	uv
Web framework	FastAPI + uvicorn
Validation	Pydantic v2 + pydantic-settings
LLM	Anthropic Claude (Haiku / Sonnet / Opus) with streaming, tool use, and prompt caching
Embedding	OpenAI text-embedding-3-small (1536-dim)
Sparse retrieval	Qdrant BM25 via fastembed
Reranker	Cohere rerank-v3.5
Vector database	Qdrant (named dense + sparse vectors, RRF fusion)
Relational database	PostgreSQL 16 + SQLAlchemy 2.0 async + Alembic
SQL safety	sqlglot AST analysis
Logging	structlog (JSON, correlation IDs via contextvars)
Tests	pytest with unit / integration / eval markers
Lint / type	ruff + mypy (strict)
Container	Docker Compose

Quick start

# 1. Install dependencies into a project-local .venv
uv sync
# 2. Copy env template and provide credentials
cp .env.example .env
# Required: APP_OPENAI_API_KEY, APP_COHERE_API_KEY, APP_ANTHROPIC_API_KEY# Optional: APP_ADMIN_TOKEN (enables /v1/admin/* endpoints)# 3. Start dependencies
docker compose up -d postgres qdrant
# 4. Apply migrations
uv run alembic upgrade head
# 5. Run the API (foreground; open a second terminal for subsequent commands)
uv run uvicorn app.main:app --reload --port 8000

Verify the service:

curl http://localhost:8000/healthz
# → {"status":"ok"}

Seed demo data

The repository ships with a small demo workspace covering four documents and a synthetic schema of users, products, and orders for the text-to-SQL tool.

uv run python scripts/seed_postgres.py --workspace demo
uv run python scripts/ingest_sample_docs.py --workspace demo

Chat

WS=$(curl -s http://localhost:8000/v1/workspaces | jq -r '.[] | select(.slug=="demo") | .id')# Document-grounded answer
curl -N -X POST "http://localhost:8000/v1/workspaces/$WS/chat" \
-H 'Content-Type: application/json' \
-d '{"message":"What is the vacation policy?"}'# Database query
curl -N -X POST "http://localhost:8000/v1/workspaces/$WS/chat" \
-H 'Content-Type: application/json' \
-d '{"message":"How many orders did user 42 place this month?"}'# Parallel tools in one turn
curl -N -X POST "http://localhost:8000/v1/workspaces/$WS/chat" \
-H 'Content-Type: application/json' \
-d '{"message":"What is the refund policy and how much has user 42 spent?"}'

Responses stream as Server-Sent Events. The final done event carries the answer, citations, iteration count, cost in USD, and the session id (use it as session_id on follow-up calls to continue a conversation).

Debug retrieval

Skip the LLM and inspect the chunks returned by hybrid search + rerank:

curl -s -X POST "http://localhost:8000/v1/workspaces/$WS/search" \
-H 'Content-Type: application/json' \
-d '{"query":"vacation policy","min_score":0.3}'| jq .

Admin usage report

curl -s "http://localhost:8000/v1/admin/usage" \
-H "X-Admin-Token: $APP_ADMIN_TOKEN"| jq .# {totals, by_workspace[], by_model[]}

Architecture

Layered architecture with one-way dependency flow:

API Layer (FastAPI controllers) → app/api/v1/
↓
Service Layer (business logic) → app/services/
↓
Tools Layer (Template Method) → app/tools/
↓
Repositories | Adapters → app/repositories/, app/adapters/
↓ ↓
Postgres, Qdrant | OpenAI, Cohere, Anthropic

External SDKs are isolated behind adapter ports (EmbedderPort, RerankerPort, LlmPort) so concrete implementations are swappable. The full request lifecycle for indexing, retrieval, and chat is documented in docs/PIPELINE.md.

API surface

Endpoint	Method	Purpose
/healthz, /readyz	GET	Liveness / readiness probes
/v1/workspaces	POST, GET	Create / list workspaces
/v1/workspaces/{id}	GET, DELETE	Read / delete a workspace
/v1/workspaces/{id}/documents	POST	Upload and index a document (multipart)
/v1/workspaces/{id}/search	POST	Hybrid retrieval (no LLM)
/v1/workspaces/{id}/chat	POST	SSE streaming chat with the agent
/v1/admin/usage	GET	Aggregated usage report (token auth)

OpenAPI spec is auto-generated at /docs (Swagger UI) and /openapi.json.

Project layout

support-platform/
├── app/
│ ├── main.py # FastAPI factory, lifespan, router registration
│ ├── core/ # settings, logging, exceptions, middleware, DI
│ ├── api/v1/ # HTTP routes
│ ├── schemas/ # Pydantic DTOs
│ ├── services/ # business logic
│ ├── adapters/ # external SDK wrappers
│ ├── repositories/ # data access (Postgres, Qdrant)
│ ├── tools/ # agent tools (ToolBase + four concrete tools)
│ ├── prompts/ # versioned prompt files
│ └── db/ # SQLAlchemy models + session
├── alembic/versions/ # database migrations
├── sample_data/ # demo documents + seed.sql
├── scripts/ # CLI helpers (ingest, seed, eval)
├── tests/ # unit / integration / eval
└── docs/ # design notes (PIPELINE, PATTERNS, ...)

Development

# Format + lint
uv run ruff format .
uv run ruff check .# Type check
uv run mypy app/
# Tests
uv run pytest -m unit -v # fast, no external deps
uv run pytest -m integration -v # requires docker compose + .env
uv run python scripts/run_eval.py --workspace demo # eval over golden set

License

MIT