Repository files navigation

DocSage

DocSage is an agentic document-intelligence system: ingest financial and business documents, persist structured rows (transactions) in a database, build a semantic index for question answering, and expose both through a FastAPI backend and a Next.js UI. The “brain” for generation is Google Gemini; retrieval uses local FAISS + sentence-transformers; multi-step reasoning is orchestrated with LangGraph.


Table of contents


What DocSage does

  1. Upload documents (PDF, images, etc.) via the API or UI.
  2. Parse them through an IDP-style pipeline: text extraction (PDF and/or OCR), heuristic classification, and structured field extraction.
  3. StoreDocument rows plus derived Transaction rows in PostgreSQL or SQLite.
  4. Index in FAISS for RAG: per document, an extraction_summary chunk (from structured extracted_data) when present, plus chunked raw_text—each chunk carries chunk_type, chunk_index, and document metadata for grounding in API responses.
  5. Answer questions via POST /api/v1/chat/insights: a LangGraph workflow can route between fast analytic paths, SQL over documents + transactions (with JOIN on document_id), and full retrieve → rerank → grade → synthesize agentic RAG with HyDE-style query rewriting. Optional history on the request enables multi-turn chat (last 20 turns used server-side); synthesis prompts insist on filename / document_id citations when evidence exists.
flowchart LR
subgraph client [Client]
Browser[Browser_Next.js]
end
subgraph api [FastAPI_api]
Routes[routers]
Agent[LangGraph_agent]
RAG[RAGService_FAISS]
DB[(SQLAlchemy_DB)]
LLM[Gemini_API]
end
Browser -->|HTTP_JSON| Routes
Routes --> Agent
Agent --> RAG
Agent --> DB
Agent --> LLM
Routes --> DB
Loading

Repository map

PathRole
api/Python package app: FastAPI entry api/app/main.py, routers, services, agents, models, api/requirements.txt, api/Dockerfile.
api/app/routers/HTTP route modules (analytics, documents, chat, anomalies, compare, receipts, export, insights report).
api/app/services/Business logic: RAG, LLM wrapper, IDP pipeline, SQL tools, extraction_summary helper (extraction_summary.py), insights, anomalies, export, etc.
api/app/agents/LangGraph graph compiler, nodes, state, langgraph_runner.py.
api/app/vectorstore/faiss_store.py: FAISS + embeddings.
api/scripts/Operational scripts (seed, ingest, embeddings, diagnostics)—not started automatically.
web/Next.js 14 App Router UI; entry layout web/src/app/layout.tsx, pages under web/src/app/.
web/src/lib/api.tsTyped fetch helpers against /api/v1.
scripts/run-dev.shLocal dev: optional Postgres, API + web.
docker-compose.postgres.ymlPostgres-only Compose (used by dev script).
docker-compose.ymlPostgres + API image.
docker-compose.prod.ymlProduction-oriented Compose example.
railway.tomlRailway: Dockerfile build, uvicorn start, /health.
.env.exampleTemplate for api/.env (copy into api/).
data/ (under api/ at runtime)Uploads and indexes: raw_docs, embeddings, processed—paths come from config.py (RAW_DOCS_PATH, FAISS_*, etc.) relative to the API working directory. Do not commit large api/data/ trees; keep them local or on a volume.

Quick start (local)

Prerequisites: Node.js, Python 3.11+, optional Docker for Postgres, Google AI Studio API key for LLM features.

From the repository root:

./scripts/run-dev.sh
  • Uses docker-compose.postgres.yml to start Postgres when Docker is available. If Docker is not running, the API and web still start; use USE_SQLITE=true in api/.env for a file DB, or start Docker and rerun.
  • Copies .env.exampleapi/.env once if missing; ensures web/.env.local has NEXT_PUBLIC_API_URL.
  • Recreates api/.venv if broken; installs Python and npm deps; runs uvicorn and npm run dev.

Ports:API_PORT (default 8000), WEB_PORT (default 3000). UI: http://localhost:3000; OpenAPI: http://localhost:8000/docs.

Busy Postgres port:POSTGRES_PORT=5433 ./scripts/run-dev.sh and set POSTGRES_PORT=5433 in api/.env.

Skip Docker:./scripts/run-dev.sh --no-docker

Database initialization

On API startup, api/app/main.py runs a lifespan hook that calls init_db(): SQLAlchemy create_all for models in api/app/models.py. This creates missing tables (e.g. documents, transactions) but does not run Alembic-style migrations for schema changes.


Configuration

API (api/app/config.py + api/.env)

Settings load from environment and optional api/.env. Unknown keys are ignored (extra="ignore") so stale variables do not crash startup.

VariablePurpose
POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB, POSTGRES_HOST, POSTGRES_PORTPostgreSQL connection when USE_SQLITE is false.
USE_SQLITEIf true, DATABASE_URL is SQLite (sqlite:///./docsage.db).
API_HOST, API_PORTUvicorn bind (used when running app.main as __main__).
GOOGLE_API_KEYGemini API key; alias GEMINI_API_KEY.
GOOGLE_AI_MODELPrimary generateContent model id.
GOOGLE_AI_MODEL_FALLBACKSComma-separated fallback model ids (overload / transient errors / unknown primary).
FAISS_INDEX_PATH, FAISS_DOCUMENTS_PATHPaths to FAISS index file and pickle sidecar for chunk metadata.
EMBEDDING_MODELsentence-transformers model name (default all-MiniLM-L6-v2, 384-d vectors).
RAW_DOCS_PATH, PROCESSED_PATHUpload and processed file roots.
CORS_ORIGINSComma-separated browser origins, or * (dev only; avoid in production).
MAX_UPLOAD_MBUpload size cap for document uploads.
DEBUG, LOG_LEVELApp logging / debug flags.

Web

VariablePurpose
NEXT_PUBLIC_API_URLOrigin of the FastAPI server (no trailing slash), e.g. http://127.0.0.1:8000. Used by web/src/lib/api.ts. Must match the host you use in the browser (localhost vs 127.0.0.1) to avoid CORS/preflight issues.
NEXT_PUBLIC_GOOGLE_OAUTH_ENABLEDOptional. Set to true to show “Continue with Google” before the client fetches GET /api/v1/auth/config; otherwise the UI reads that endpoint and only shows the button when the API has OAuth credentials.
NEXT_PUBLIC_SITE_URLOptional canonical site URL for metadata. If unset on Vercel, VERCEL_URL is used in web/src/app/layout.tsx for metadataBase.

Docker Compose: pass GOOGLE_API_KEY, GOOGLE_AI_MODEL, GOOGLE_AI_MODEL_FALLBACKS into the api service (see docker-compose.yml, docker-compose.prod.yml).


Backend architecture

FastAPI application

api/app/main.py constructs the app with:

  • CORS from settings.cors_origins_list.
  • APIRouter subtree mounted at /api/v1 including analytics, anomalies, documents, compare, receipts, export, insights report, chat.
  • LegacyPOST /chat/insights delegating to the same handler as v1 chat.

Persistence

  • SQLAlchemy 2.x engine + SessionLocal in api/app/db.py.
  • Models in api/app/models.py:
    • Document: filename, path, type, raw text, JSON extracted_data, timestamps.
    • Transaction: document_id, date, amount, vendor, category, description, JSON metadata (ORM attribute meta_data to avoid reserved name issues), confidence / correction flags.
    • DocumentCorrection: audit of user corrections.

Routers use Depends(get_db) for request-scoped sessions.

Validation

api/app/schemas.py defines Pydantic models for HTTP I/O. Notable:

  • QueryRequest: query, use_rag, use_sql, optional history (List[ChatMessage] with role and content). The chat router keeps the last 20 turns with non-empty content. use_rag / use_sql bias LangGraph routing (e.g. SQL-only path when RAG is off and keywords suggest aggregation).

Concept glossary

Each item: what the concept is, then how DocSage applies it (files).

Intelligent Document Processing (IDP)

IDP is the class of systems that turn messy documents (PDFs, scans) into structured, machine-usable data—classification, key-value extraction, validation—not just raw text.

DocSage:api/app/services/idp_pipeline.py implements extraction and heuristic classification; the upload path in api/app/routers/documents.py calls parse_document then persists rows. This is “IDP-inspired”: rules + LLM/heuristics rather than a full enterprise IDP product.

OCR (Optical Character Recognition)

OCR recovers text from pixels (photos, scanned pages).

DocSage:pytesseract with Pillow-compatible inputs in extract_text_with_ocr (idp_pipeline.py). The Docker image installs tesseract-ocr (api/Dockerfile) so containers can OCR without extra host setup.

PDF text extraction

Digital PDFs often expose a text layer; extraction without OCR is faster and more accurate.

DocSage:pdfplumber in extract_text_from_pdf walks pages and concatenates extract_text() output (idp_pipeline.py). Image-only PDFs may still need rasterization + OCR (pipeline-dependent).

Heuristic classification and regex extraction

Heuristics use keywords and patterns to guess document type and pull amounts, dates, and vendors without a dedicated ML model per field.

DocSage:classify_document, extract_amounts, and related helpers in idp_pipeline.py; extracted JSON is stored on Document.extracted_data and Transaction rows are synthesized via extract_transactions_from_document (api/scripts/ingest_docs.py) used from the documents router.

Embeddings

An embedding is a dense vector representing text (or other modalities) in a space where semantic similarityvector proximity.

DocSage:SentenceTransformer in api/app/vectorstore/faiss_store.py encodes strings; default model all-MiniLM-L6-v2 produces 384-dimensional vectors (EMBEDDING_MODEL in config). api/scripts/build_embeddings.py walks all Document rows and, for each, emits (1) a single extraction_summary string via extraction_to_index_text (extraction_summary.py) when extracted_data is present, and (2) sliding-window chunks over raw_text when present. Metadata on each vector row includes chunk_type (extraction_summary vs raw_text), chunk_index, total_chunks (for raw splits), and document id/filename—used downstream for rerank sources and UI citations.

Vector store and approximate search (FAISS)

A vector store indexes vectors for nearest-neighbor search (which chunks are closest to the query embedding).

DocSage:FAISSIndexFlatL2—exact L2 search over all vectors (simple, no training). Index and parallel pickle list of metadata are saved to FAISS_INDEX_PATH / FAISS_DOCUMENTS_PATH (relative to API cwd, typically api/data/embeddings/ in local dev). RAGService loads the index if files exist and exposes search(query, k).

RAG (Retrieval-Augmented Generation)

RAG grounds LLM answers in retrieved passages from a corpus instead of parametric memory alone, reducing hallucination on factual questions about your documents.

DocSage:RAGService wraps the FAISS store; LangGraph nodes call search with a rewritten query after HyDE and optional reranking (api/app/agents/langgraph/nodes.py). node_rerank / node_synthesize attach sources: document_id, filename, chunk_index, chunk_type, score for citation-style UX; final synthesis instructs the model to cite filename and document id when evidence exists and to avoid inventing facts when context and SQL are empty.

LangGraph and agentic control flow

LangGraph models an agent as a state machine: nodes (functions) update state; edges (conditional or fixed) choose the next step. “Agentic” here means multiple LLM and tool steps with branching, not a single prompt.

DocSage:build_agent_graph compiles a StateGraph over AgentState (api/app/agents/langgraph/state.py) including optional history for multi-turn prompts. run_agent_pipeline invokes the compiled graph with graph.invoke, then shapes the response for the REST API.

LangChain packages in api/requirements.txt (langchain-core, langchain-community) support the broader ecosystem; application code under app/ imports LangGraph directly rather than high-level LangChain chains.

HyDE (Hypothetical Document Embeddings)

HyDE asks the LLM to draft a fake answer or passage that would answer the question; that text is embedded and used to retrieve real chunks. It often improves recall when the raw user question is short or mismatched to chunk wording.

DocSage:node_hyde_rewrite in nodes.py calls call_llm to produce a hypothetical block; retrieval uses the rewritten text (see also refinement_hint on failed grades). HyDE, grading, SQL generation, and synthesis prompts also receive a compact conversation block from history when the client sends prior turns.

Retrieval depth, cross-encoder reranking, and grading loop

Two-stage retrieval can mean: (1) cheap bi-encoder over many candidates, then (2) cross-encoder scoring query–passage pairs for a top subset. A grader decides if context is good enough or triggers another retrieval loop.

DocSage: constants RETRIEVE_K, RERANK_POOL, RERANK_KEEP, MAX_RETRIEVAL_LOOPS in nodes.py. cross-encoder/ms-marco-MiniLM-L-6-v2 scores pairs. node_grade sets grade_pass; conditional edges in graph.py send failures back to hyde until the cap, then proceed to optional_sqlsynthesize.

SQL grounding (LLM-generated SELECT)

Grounding here means the LLM sees the real table schema and sample rows before emitting read-only SQL executed against your DB.

DocSage:SQLTools introspects transactions for low-level helpers and exposes get_multitable_sql_llm_context()—combined documents + transactions schemas, truncated raw_text previews, and shrunk extracted_data in document samples (SQLite vs Postgres aware). _generate_sql in nodes.py includes that context, the user question, and a short conversation block from history, and explains that transactions.document_id references documents.id (JOIN allowed). node_sql_only and node_optional_sql merge SQL results into answers; optional SQL is keyword-gated (including document/invoice-style terms).

Fast analytic path (metrics shortcut)

Some questions match precomputed aggregates faster than full RAG.

DocSage:node_route checks keyword hints for vendor/category breakdowns and routes to node_metrics_fast, which calls InsightsService (api/app/services/insights.py) and returns without vector search (nodes.py).

Anomaly detection

Anomaly detection flags unusual rows (duplicates, outliers, date oddities).

DocSage:api/app/services/anomaly_detection.py; exposed via api/app/routers/anomalies.py.

Google Gemini (Generative Language API)

Gemini is accessed through RESTgenerateContent (v1beta), not a proprietary SDK requirement in this repo.

DocSage:api/app/services/llm_service.py builds the request with optional systemInstruction, walks a model chain (primary + GOOGLE_AI_MODEL_FALLBACKS), retries 429 / 5xx with backoff and optional Retry-After, skips to the next model on 400 / 404 (e.g. deprecated model id), and returns assistant text or structured error strings.


LangGraph chat pipeline

Graph topology (mermaid)

Mirrors api/app/agents/langgraph/graph.py.

flowchart TD
entry[route_entry]
route[node_route]
metrics[node_metrics_fast]
sqlOnly[node_sql_only]
hyde[node_hyde_rewrite]
retrieve[node_retrieve]
rerank[node_rerank]
grade[node_grade]
sqlOpt[node_optional_sql]
synth[node_synthesize]
endNode[END]
entry --> route
route -->|metrics_fast| metrics
route -->|sql_only| sqlOnly
route -->|agentic_rag| hyde
hyde --> retrieve
retrieve --> rerank
rerank --> grade
grade -->|retry_HyDE_loop| hyde
grade -->|pass_or_cap| sqlOpt
sqlOpt --> synth
metrics --> endNode
sqlOnly --> endNode
synth --> endNode
Loading

The grade → hyde edge is conditional: only when grade_pass is false and retrieval_iteration is below MAX_RETRIEVAL_LOOPS (_grade_next).

HTTP entry and response shape

api/app/routers/chat.py:

  • Lazily constructs a singleton RAGService (loads FAISS if index files exist).
  • run_chat maps request.history to the graph (last 20 non-empty turns).
  • run_chat calls run_agent_pipeline(query, rag, use_rag=..., use_sql=..., history=...).
  • Returns QueryResponse: answer, sources (list of dicts with document_id, filename, chunk_index, chunk_type, score when RAG ran), sql_query, steps, tool_calls (derived from steps for UI convenience).

When RAG is “skipped” in spirit:use_rag=False with SQL-biased routing yields sql_only. use_rag=False also clears sources in the runner output. use_sql=False disables the optional SQL augmentation node path in the graph state.

Other LLM call sites (outside the graph)

Several features call call_llm directly without LangGraph: e.g. insights report generation (api/app/services/insights_generator.py), categorization (api/app/services/categorization.py), HyDE / synthesize / SQL prompt nodes. The graph is the orchestrator for interactive chat; batch/report flows may be linear.


Document ingestion and data flow

  1. POST/api/v1/documents with multipart file (documents.py).

  2. File bytes saved under RAW_DOCS_PATH.

  3. parse_document(path) runs the IDP pipeline (idp_pipeline.py).

  4. Document inserted; extract_transactions_from_document yields dicts → Transaction rows committed.

  5. FAISS is not automatically rebuilt on every upload. After new imports, extraction changes, or IDP tweaks, refresh vectors so extraction_summary and raw_text chunks stay in sync—for example:

    cd api && ./.venv/bin/python scripts/build_embeddings.py

    You can also use RAGService.build_index / add_documents (rag.py) in custom ops; add_documents currently rebuilds the full index for simplicity.


HTTP API surface

All v1 routes are prefixed with /api/v1 unless noted.

TagMethodPathPurpose
analyticsGET/analytics/summaryDashboard KPIs: counts, spend, averages.
GET/analytics/time-seriesTime-bucketed series for charts.
GET/analytics/vendor-statsTop vendors by spend.
GET/analytics/category-breakdownSpend by category.
GET/analytics/spending-forecastSimple forward-looking projection.
GET/analytics/monthly-spendSpend for a given year/month.
anomaliesGET/anomaliesRule-based anomaly list.
chatPOST/chat/insightsAgentic RAG + SQL pipeline. Body: query (required), use_rag, use_sql, optional history (array of { role, content }, server uses last 20 non-empty turns).
compareGET/documents/{document_id}/similarSimilar documents (e.g. shared vendor / join logic in service).
POST/documents/comparePairwise diff / compare (CompareBody).
documentsGET/documentsList documents (filters, pagination).
POST/documentsUpload + parse + persist transactions.
GET/documents/{id}Metadata.
GET/documents/{id}/detailRich detail payload.
GET/documents/{id}/confidenceExtraction confidence signals.
GET/documents/{id}/previewPreview / annotated stream where implemented.
PATCH/documents/{id}Update extracted JSON (DocumentUpdateBody).
exportsGET/exports/excelDownload Excel export blob.
GET/exports/summaryText/markdown summary for export UX.
insights-reportPOST/insights/generate-reportLLM-generated narrative report.
receipt-matchingGET/receipt-matching/unmatchedQueue of unmatched receipts.
POST/receipt-matching/{receipt_doc_id}/matchLink receipt to candidate transaction.

Legacy (no /api/v1 prefix):POST /chat/insights — same body/response as v1 chat.

Interactive docs: /docs (Swagger UI).


Frontend (web)

Pages (examples):dashboard, chat, documents, insights, anomalies, compare, export, receipt-matching; marketing home composes LandingStory.


Dependencies, scripts, and containers

Python (api/requirements.txt) — grouped by role

GroupExamples
HTTPfastapi, uvicorn, python-multipart, pydantic, pydantic-settings
DBsqlalchemy, psycopg2-binary
Agentslanggraph, langchain-core, langchain-community
Vectors / MLfaiss-cpu, sentence-transformers, numpy, pandas
Documentspdfplumber, pytesseract, Pillow, opencv-python, openpyxl
LLM HTTPrequests

Operational scripts (api/scripts/)

Not invoked by default: seed_db.py, ingest_docs.py, build_embeddings.py, migrate_database.py, add_documents_from_folder.py, diagnose_and_fix_transactions.py, download_huggingface_dataset.py, preload_kaggle_invoices.py, etc. Use them manually for migrations, backfills, demo data, and embedding rebuilds.

Docker

api/DockerfileCOPYs from repo root: docker build -f api/Dockerfile . installs Tesseract system packages, Python deps, copies api/app and api/scripts, creates data/ subtrees, runs uvicorn on port 8000.

Compose files wire Postgres + env; see repository root YAMLs.

Railway

railway.toml: Dockerfile builder, uvicorn app.main:app --host 0.0.0.0 --port $PORT, health check /health, restart policy.


Deployment

  • Frontend: deploy subdirectory web/ (e.g. Vercel). Set NEXT_PUBLIC_API_URL to your API’s public origin. Set NEXT_PUBLIC_SITE_URL or rely on VERCEL_URL for metadata (see layout).
  • Backend: container host (Fly, Railway, Cloud Run, etc.) using api/Dockerfile with root build context. Inject GOOGLE_API_KEY, DB URL, CORS_ORIGINS matching the exact browser origin(s) in production.
  • Persistence: mount a volume (or object storage strategy) for api/data/raw_docs, api/data/embeddings, and the SQLite file if used—ephemeral disks lose indexes and uploads on restart.

Authentication and multi-tenancy

DocSage uses JWT-based authentication with email/password registration and optional Google OAuth.

Backend

  • User model in api/app/models.py: email (unique), hashed password (nullable for OAuth-only), optional oauth_provider/oauth_sub.
  • Auth router at /api/v1/auth/ (api/app/routers/auth.py):
    • POST /register — email + password; returns JWT.
    • POST /login — email + password; returns JWT.
    • GET /me — current user from token.
    • GET /google — redirects to Google consent screen.
    • GET /google/callback — exchanges code, upserts user, redirects to frontend with token in URL hash.
  • Dependencyget_current_user in api/app/deps.py protects all non-auth routes.
  • Tenant isolation: every query in documents, transactions, analytics, anomalies, compare, receipts, export, reports, and chat is filtered by user_id.
  • Per-user upload paths: files are stored under data/raw_docs/user_{id}/.
  • Per-user RAG: FAISS indexes live at data/embeddings/user_{id}/.
  • Chat sessions API at /api/v1/chat/sessions (CRUD scoped to current user).

Frontend

  • AuthProvider in web/src/contexts/auth.tsx: stores JWT in localStorage, exposes login, register, logout, setTokenFromOAuth.
  • Bearer token added to all API requests via web/src/lib/api.ts.
  • Route protection in ShellLayout: unauthenticated users are redirected to /login; public routes: /, /login, /register, /auth/callback.
  • Login/Register pages: Google sign-in is shown only when the API reports OAuth is enabled (GET /auth/config) or NEXT_PUBLIC_GOOGLE_OAUTH_ENABLED=true.
  • Chat sessions sync to the server API when authenticated; fall back to localStorage when offline.

Configuration

VariablePurpose
JWT_SECRETSecret for HS256 token signing (change in production).
JWT_EXPIRE_MINUTESToken validity (default 7 days).
GOOGLE_OAUTH_CLIENT_IDGoogle Cloud console client ID (optional).
GOOGLE_OAUTH_CLIENT_SECRETMatching secret.
GOOGLE_OAUTH_REDIRECT_URIMust match console; default http://localhost:8000/api/v1/auth/google/callback.
FRONTEND_URLWhere the OAuth callback redirects with the token hash fragment.

Public GET /api/v1/auth/config (no auth): returns { "google_oauth_enabled": boolean } so the web UI can hide “Continue with Google” when OAuth is not configured on the server.

Migration

Run python scripts/migrate_database.py from api/ to add user_id columns to existing tables and create users / chat_sessions tables. Existing rows without a user_id are hidden from authenticated queries until backfilled.


Limitations and extension points

AreaLimitationPossible extension
FAISSIndexFlatL2 is linear; slow at very large NIVF / HNSW, or managed vector DB (Pinecone, pgvector, …).
Index updatesadd_documents rebuilds whole indexIncremental add, background jobs, versioning.
Schemacreate_all only; no Alembic in treeMigrations for production schema evolution.
Chathistory trimmed to 20 turns with content (chat.py)Configurable cap, thread storage, or rolling summary.
OCRHost must have Tesseract unless using Docker imageCloud OCR APIs, better layout models.
Compare routesMounted at /documents/... alongside document CRUDEnsure route ordering in OpenAPI matches FastAPI resolution for edge IDs.
SecretsNever commit real GOOGLE_API_KEY; rotate if leakedSecret manager, .env gitignored (already).

This README is the authoritative high-level map of the codebase; for line-level behavior, follow the links into api/app/ and web/src/.

About

AI-powered Intelligent Document Processing (IDP) system with RAG, anomaly detection, and natural language insights. Local, zero-cost alternative to AWS Textract + Bedrock.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Repository files navigation

DocSage

DocSage is an agentic document-intelligence system: ingest financial and business documents, persist structured rows (transactions) in a database, build a semantic index for question answering, and expose both through a FastAPI backend and a Next.js UI. The “brain” for generation is Google Gemini; retrieval uses local FAISS + sentence-transformers; multi-step reasoning is orchestrated with LangGraph.


Table of contents


What DocSage does

  1. Upload documents (PDF, images, etc.) via the API or UI.
  2. Parse them through an IDP-style pipeline: text extraction (PDF and/or OCR), heuristic classification, and structured field extraction.
  3. StoreDocument rows plus derived Transaction rows in PostgreSQL or SQLite.
  4. Index in FAISS for RAG: per document, an extraction_summary chunk (from structured extracted_data) when present, plus chunked raw_text—each chunk carries chunk_type, chunk_index, and document metadata for grounding in API responses.
  5. Answer questions via POST /api/v1/chat/insights: a LangGraph workflow can route between fast analytic paths, SQL over documents + transactions (with JOIN on document_id), and full retrieve → rerank → grade → synthesize agentic RAG with HyDE-style query rewriting. Optional history on the request enables multi-turn chat (last 20 turns used server-side); synthesis prompts insist on filename / document_id citations when evidence exists.
flowchart LR
subgraph client [Client]
Browser[Browser_Next.js]
end
subgraph api [FastAPI_api]
Routes[routers]
Agent[LangGraph_agent]
RAG[RAGService_FAISS]
DB[(SQLAlchemy_DB)]
LLM[Gemini_API]
end
Browser -->|HTTP_JSON| Routes
Routes --> Agent
Agent --> RAG
Agent --> DB
Agent --> LLM
Routes --> DB
Loading

Repository map

PathRole
api/Python package app: FastAPI entry api/app/main.py, routers, services, agents, models, api/requirements.txt, api/Dockerfile.
api/app/routers/HTTP route modules (analytics, documents, chat, anomalies, compare, receipts, export, insights report).
api/app/services/Business logic: RAG, LLM wrapper, IDP pipeline, SQL tools, extraction_summary helper (extraction_summary.py), insights, anomalies, export, etc.
api/app/agents/LangGraph graph compiler, nodes, state, langgraph_runner.py.
api/app/vectorstore/faiss_store.py: FAISS + embeddings.
api/scripts/Operational scripts (seed, ingest, embeddings, diagnostics)—not started automatically.
web/Next.js 14 App Router UI; entry layout web/src/app/layout.tsx, pages under web/src/app/.
web/src/lib/api.tsTyped fetch helpers against /api/v1.
scripts/run-dev.shLocal dev: optional Postgres, API + web.
docker-compose.postgres.ymlPostgres-only Compose (used by dev script).
docker-compose.ymlPostgres + API image.
docker-compose.prod.ymlProduction-oriented Compose example.
railway.tomlRailway: Dockerfile build, uvicorn start, /health.
.env.exampleTemplate for api/.env (copy into api/).
data/ (under api/ at runtime)Uploads and indexes: raw_docs, embeddings, processed—paths come from config.py (RAW_DOCS_PATH, FAISS_*, etc.) relative to the API working directory. Do not commit large api/data/ trees; keep them local or on a volume.

Quick start (local)

Prerequisites: Node.js, Python 3.11+, optional Docker for Postgres, Google AI Studio API key for LLM features.

From the repository root:

./scripts/run-dev.sh
  • Uses docker-compose.postgres.yml to start Postgres when Docker is available. If Docker is not running, the API and web still start; use USE_SQLITE=true in api/.env for a file DB, or start Docker and rerun.
  • Copies .env.exampleapi/.env once if missing; ensures web/.env.local has NEXT_PUBLIC_API_URL.
  • Recreates api/.venv if broken; installs Python and npm deps; runs uvicorn and npm run dev.

Ports:API_PORT (default 8000), WEB_PORT (default 3000). UI: http://localhost:3000; OpenAPI: http://localhost:8000/docs.

Busy Postgres port:POSTGRES_PORT=5433 ./scripts/run-dev.sh and set POSTGRES_PORT=5433 in api/.env.

Skip Docker:./scripts/run-dev.sh --no-docker

Database initialization

On API startup, api/app/main.py runs a lifespan hook that calls init_db(): SQLAlchemy create_all for models in api/app/models.py. This creates missing tables (e.g. documents, transactions) but does not run Alembic-style migrations for schema changes.


Configuration

API (api/app/config.py + api/.env)

Settings load from environment and optional api/.env. Unknown keys are ignored (extra="ignore") so stale variables do not crash startup.

VariablePurpose
POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB, POSTGRES_HOST, POSTGRES_PORTPostgreSQL connection when USE_SQLITE is false.
USE_SQLITEIf true, DATABASE_URL is SQLite (sqlite:///./docsage.db).
API_HOST, API_PORTUvicorn bind (used when running app.main as __main__).
GOOGLE_API_KEYGemini API key; alias GEMINI_API_KEY.
GOOGLE_AI_MODELPrimary generateContent model id.
GOOGLE_AI_MODEL_FALLBACKSComma-separated fallback model ids (overload / transient errors / unknown primary).
FAISS_INDEX_PATH, FAISS_DOCUMENTS_PATHPaths to FAISS index file and pickle sidecar for chunk metadata.
EMBEDDING_MODELsentence-transformers model name (default all-MiniLM-L6-v2, 384-d vectors).
RAW_DOCS_PATH, PROCESSED_PATHUpload and processed file roots.
CORS_ORIGINSComma-separated browser origins, or * (dev only; avoid in production).
MAX_UPLOAD_MBUpload size cap for document uploads.
DEBUG, LOG_LEVELApp logging / debug flags.

Web

VariablePurpose
NEXT_PUBLIC_API_URLOrigin of the FastAPI server (no trailing slash), e.g. http://127.0.0.1:8000. Used by web/src/lib/api.ts. Must match the host you use in the browser (localhost vs 127.0.0.1) to avoid CORS/preflight issues.
NEXT_PUBLIC_GOOGLE_OAUTH_ENABLEDOptional. Set to true to show “Continue with Google” before the client fetches GET /api/v1/auth/config; otherwise the UI reads that endpoint and only shows the button when the API has OAuth credentials.
NEXT_PUBLIC_SITE_URLOptional canonical site URL for metadata. If unset on Vercel, VERCEL_URL is used in web/src/app/layout.tsx for metadataBase.

Docker Compose: pass GOOGLE_API_KEY, GOOGLE_AI_MODEL, GOOGLE_AI_MODEL_FALLBACKS into the api service (see docker-compose.yml, docker-compose.prod.yml).


Backend architecture

FastAPI application

api/app/main.py constructs the app with:

  • CORS from settings.cors_origins_list.
  • APIRouter subtree mounted at /api/v1 including analytics, anomalies, documents, compare, receipts, export, insights report, chat.
  • LegacyPOST /chat/insights delegating to the same handler as v1 chat.

Persistence

  • SQLAlchemy 2.x engine + SessionLocal in api/app/db.py.
  • Models in api/app/models.py:
    • Document: filename, path, type, raw text, JSON extracted_data, timestamps.
    • Transaction: document_id, date, amount, vendor, category, description, JSON metadata (ORM attribute meta_data to avoid reserved name issues), confidence / correction flags.
    • DocumentCorrection: audit of user corrections.

Routers use Depends(get_db) for request-scoped sessions.

Validation

api/app/schemas.py defines Pydantic models for HTTP I/O. Notable:

  • QueryRequest: query, use_rag, use_sql, optional history (List[ChatMessage] with role and content). The chat router keeps the last 20 turns with non-empty content. use_rag / use_sql bias LangGraph routing (e.g. SQL-only path when RAG is off and keywords suggest aggregation).

Concept glossary

Each item: what the concept is, then how DocSage applies it (files).

Intelligent Document Processing (IDP)

IDP is the class of systems that turn messy documents (PDFs, scans) into structured, machine-usable data—classification, key-value extraction, validation—not just raw text.

DocSage:api/app/services/idp_pipeline.py implements extraction and heuristic classification; the upload path in api/app/routers/documents.py calls parse_document then persists rows. This is “IDP-inspired”: rules + LLM/heuristics rather than a full enterprise IDP product.

OCR (Optical Character Recognition)

OCR recovers text from pixels (photos, scanned pages).

DocSage:pytesseract with Pillow-compatible inputs in extract_text_with_ocr (idp_pipeline.py). The Docker image installs tesseract-ocr (api/Dockerfile) so containers can OCR without extra host setup.

PDF text extraction

Digital PDFs often expose a text layer; extraction without OCR is faster and more accurate.

DocSage:pdfplumber in extract_text_from_pdf walks pages and concatenates extract_text() output (idp_pipeline.py). Image-only PDFs may still need rasterization + OCR (pipeline-dependent).

Heuristic classification and regex extraction

Heuristics use keywords and patterns to guess document type and pull amounts, dates, and vendors without a dedicated ML model per field.

DocSage:classify_document, extract_amounts, and related helpers in idp_pipeline.py; extracted JSON is stored on Document.extracted_data and Transaction rows are synthesized via extract_transactions_from_document (api/scripts/ingest_docs.py) used from the documents router.

Embeddings

An embedding is a dense vector representing text (or other modalities) in a space where semantic similarityvector proximity.

DocSage:SentenceTransformer in api/app/vectorstore/faiss_store.py encodes strings; default model all-MiniLM-L6-v2 produces 384-dimensional vectors (EMBEDDING_MODEL in config). api/scripts/build_embeddings.py walks all Document rows and, for each, emits (1) a single extraction_summary string via extraction_to_index_text (extraction_summary.py) when extracted_data is present, and (2) sliding-window chunks over raw_text when present. Metadata on each vector row includes chunk_type (extraction_summary vs raw_text), chunk_index, total_chunks (for raw splits), and document id/filename—used downstream for rerank sources and UI citations.

Vector store and approximate search (FAISS)

A vector store indexes vectors for nearest-neighbor search (which chunks are closest to the query embedding).

DocSage:FAISSIndexFlatL2—exact L2 search over all vectors (simple, no training). Index and parallel pickle list of metadata are saved to FAISS_INDEX_PATH / FAISS_DOCUMENTS_PATH (relative to API cwd, typically api/data/embeddings/ in local dev). RAGService loads the index if files exist and exposes search(query, k).

RAG (Retrieval-Augmented Generation)

RAG grounds LLM answers in retrieved passages from a corpus instead of parametric memory alone, reducing hallucination on factual questions about your documents.

DocSage:RAGService wraps the FAISS store; LangGraph nodes call search with a rewritten query after HyDE and optional reranking (api/app/agents/langgraph/nodes.py). node_rerank / node_synthesize attach sources: document_id, filename, chunk_index, chunk_type, score for citation-style UX; final synthesis instructs the model to cite filename and document id when evidence exists and to avoid inventing facts when context and SQL are empty.

LangGraph and agentic control flow

LangGraph models an agent as a state machine: nodes (functions) update state; edges (conditional or fixed) choose the next step. “Agentic” here means multiple LLM and tool steps with branching, not a single prompt.

DocSage:build_agent_graph compiles a StateGraph over AgentState (api/app/agents/langgraph/state.py) including optional history for multi-turn prompts. run_agent_pipeline invokes the compiled graph with graph.invoke, then shapes the response for the REST API.

LangChain packages in api/requirements.txt (langchain-core, langchain-community) support the broader ecosystem; application code under app/ imports LangGraph directly rather than high-level LangChain chains.

HyDE (Hypothetical Document Embeddings)

HyDE asks the LLM to draft a fake answer or passage that would answer the question; that text is embedded and used to retrieve real chunks. It often improves recall when the raw user question is short or mismatched to chunk wording.

DocSage:node_hyde_rewrite in nodes.py calls call_llm to produce a hypothetical block; retrieval uses the rewritten text (see also refinement_hint on failed grades). HyDE, grading, SQL generation, and synthesis prompts also receive a compact conversation block from history when the client sends prior turns.

Retrieval depth, cross-encoder reranking, and grading loop

Two-stage retrieval can mean: (1) cheap bi-encoder over many candidates, then (2) cross-encoder scoring query–passage pairs for a top subset. A grader decides if context is good enough or triggers another retrieval loop.

DocSage: constants RETRIEVE_K, RERANK_POOL, RERANK_KEEP, MAX_RETRIEVAL_LOOPS in nodes.py. cross-encoder/ms-marco-MiniLM-L-6-v2 scores pairs. node_grade sets grade_pass; conditional edges in graph.py send failures back to hyde until the cap, then proceed to optional_sqlsynthesize.

SQL grounding (LLM-generated SELECT)

Grounding here means the LLM sees the real table schema and sample rows before emitting read-only SQL executed against your DB.

DocSage:SQLTools introspects transactions for low-level helpers and exposes get_multitable_sql_llm_context()—combined documents + transactions schemas, truncated raw_text previews, and shrunk extracted_data in document samples (SQLite vs Postgres aware). _generate_sql in nodes.py includes that context, the user question, and a short conversation block from history, and explains that transactions.document_id references documents.id (JOIN allowed). node_sql_only and node_optional_sql merge SQL results into answers; optional SQL is keyword-gated (including document/invoice-style terms).

Fast analytic path (metrics shortcut)

Some questions match precomputed aggregates faster than full RAG.

DocSage:node_route checks keyword hints for vendor/category breakdowns and routes to node_metrics_fast, which calls InsightsService (api/app/services/insights.py) and returns without vector search (nodes.py).

Anomaly detection

Anomaly detection flags unusual rows (duplicates, outliers, date oddities).

DocSage:api/app/services/anomaly_detection.py; exposed via api/app/routers/anomalies.py.

Google Gemini (Generative Language API)

Gemini is accessed through RESTgenerateContent (v1beta), not a proprietary SDK requirement in this repo.

DocSage:api/app/services/llm_service.py builds the request with optional systemInstruction, walks a model chain (primary + GOOGLE_AI_MODEL_FALLBACKS), retries 429 / 5xx with backoff and optional Retry-After, skips to the next model on 400 / 404 (e.g. deprecated model id), and returns assistant text or structured error strings.


LangGraph chat pipeline

Graph topology (mermaid)

Mirrors api/app/agents/langgraph/graph.py.

flowchart TD
entry[route_entry]
route[node_route]
metrics[node_metrics_fast]
sqlOnly[node_sql_only]
hyde[node_hyde_rewrite]
retrieve[node_retrieve]
rerank[node_rerank]
grade[node_grade]
sqlOpt[node_optional_sql]
synth[node_synthesize]
endNode[END]
entry --> route
route -->|metrics_fast| metrics
route -->|sql_only| sqlOnly
route -->|agentic_rag| hyde
hyde --> retrieve
retrieve --> rerank
rerank --> grade
grade -->|retry_HyDE_loop| hyde
grade -->|pass_or_cap| sqlOpt
sqlOpt --> synth
metrics --> endNode
sqlOnly --> endNode
synth --> endNode
Loading

The grade → hyde edge is conditional: only when grade_pass is false and retrieval_iteration is below MAX_RETRIEVAL_LOOPS (_grade_next).

HTTP entry and response shape

api/app/routers/chat.py:

  • Lazily constructs a singleton RAGService (loads FAISS if index files exist).
  • run_chat maps request.history to the graph (last 20 non-empty turns).
  • run_chat calls run_agent_pipeline(query, rag, use_rag=..., use_sql=..., history=...).
  • Returns QueryResponse: answer, sources (list of dicts with document_id, filename, chunk_index, chunk_type, score when RAG ran), sql_query, steps, tool_calls (derived from steps for UI convenience).

When RAG is “skipped” in spirit:use_rag=False with SQL-biased routing yields sql_only. use_rag=False also clears sources in the runner output. use_sql=False disables the optional SQL augmentation node path in the graph state.

Other LLM call sites (outside the graph)

Several features call call_llm directly without LangGraph: e.g. insights report generation (api/app/services/insights_generator.py), categorization (api/app/services/categorization.py), HyDE / synthesize / SQL prompt nodes. The graph is the orchestrator for interactive chat; batch/report flows may be linear.


Document ingestion and data flow

  1. POST/api/v1/documents with multipart file (documents.py).

  2. File bytes saved under RAW_DOCS_PATH.

  3. parse_document(path) runs the IDP pipeline (idp_pipeline.py).

  4. Document inserted; extract_transactions_from_document yields dicts → Transaction rows committed.

  5. FAISS is not automatically rebuilt on every upload. After new imports, extraction changes, or IDP tweaks, refresh vectors so extraction_summary and raw_text chunks stay in sync—for example:

    cd api && ./.venv/bin/python scripts/build_embeddings.py

    You can also use RAGService.build_index / add_documents (rag.py) in custom ops; add_documents currently rebuilds the full index for simplicity.


HTTP API surface

All v1 routes are prefixed with /api/v1 unless noted.

TagMethodPathPurpose
analyticsGET/analytics/summaryDashboard KPIs: counts, spend, averages.
GET/analytics/time-seriesTime-bucketed series for charts.
GET/analytics/vendor-statsTop vendors by spend.
GET/analytics/category-breakdownSpend by category.
GET/analytics/spending-forecastSimple forward-looking projection.
GET/analytics/monthly-spendSpend for a given year/month.
anomaliesGET/anomaliesRule-based anomaly list.
chatPOST/chat/insightsAgentic RAG + SQL pipeline. Body: query (required), use_rag, use_sql, optional history (array of { role, content }, server uses last 20 non-empty turns).
compareGET/documents/{document_id}/similarSimilar documents (e.g. shared vendor / join logic in service).
POST/documents/comparePairwise diff / compare (CompareBody).
documentsGET/documentsList documents (filters, pagination).
POST/documentsUpload + parse + persist transactions.
GET/documents/{id}Metadata.
GET/documents/{id}/detailRich detail payload.
GET/documents/{id}/confidenceExtraction confidence signals.
GET/documents/{id}/previewPreview / annotated stream where implemented.
PATCH/documents/{id}Update extracted JSON (DocumentUpdateBody).
exportsGET/exports/excelDownload Excel export blob.
GET/exports/summaryText/markdown summary for export UX.
insights-reportPOST/insights/generate-reportLLM-generated narrative report.
receipt-matchingGET/receipt-matching/unmatchedQueue of unmatched receipts.
POST/receipt-matching/{receipt_doc_id}/matchLink receipt to candidate transaction.

Legacy (no /api/v1 prefix):POST /chat/insights — same body/response as v1 chat.

Interactive docs: /docs (Swagger UI).


Frontend (web)

Pages (examples):dashboard, chat, documents, insights, anomalies, compare, export, receipt-matching; marketing home composes LandingStory.


Dependencies, scripts, and containers

Python (api/requirements.txt) — grouped by role

GroupExamples
HTTPfastapi, uvicorn, python-multipart, pydantic, pydantic-settings
DBsqlalchemy, psycopg2-binary
Agentslanggraph, langchain-core, langchain-community
Vectors / MLfaiss-cpu, sentence-transformers, numpy, pandas
Documentspdfplumber, pytesseract, Pillow, opencv-python, openpyxl
LLM HTTPrequests

Operational scripts (api/scripts/)

Not invoked by default: seed_db.py, ingest_docs.py, build_embeddings.py, migrate_database.py, add_documents_from_folder.py, diagnose_and_fix_transactions.py, download_huggingface_dataset.py, preload_kaggle_invoices.py, etc. Use them manually for migrations, backfills, demo data, and embedding rebuilds.

Docker

api/DockerfileCOPYs from repo root: docker build -f api/Dockerfile . installs Tesseract system packages, Python deps, copies api/app and api/scripts, creates data/ subtrees, runs uvicorn on port 8000.

Compose files wire Postgres + env; see repository root YAMLs.

Railway

railway.toml: Dockerfile builder, uvicorn app.main:app --host 0.0.0.0 --port $PORT, health check /health, restart policy.


Deployment

  • Frontend: deploy subdirectory web/ (e.g. Vercel). Set NEXT_PUBLIC_API_URL to your API’s public origin. Set NEXT_PUBLIC_SITE_URL or rely on VERCEL_URL for metadata (see layout).
  • Backend: container host (Fly, Railway, Cloud Run, etc.) using api/Dockerfile with root build context. Inject GOOGLE_API_KEY, DB URL, CORS_ORIGINS matching the exact browser origin(s) in production.
  • Persistence: mount a volume (or object storage strategy) for api/data/raw_docs, api/data/embeddings, and the SQLite file if used—ephemeral disks lose indexes and uploads on restart.

Authentication and multi-tenancy

DocSage uses JWT-based authentication with email/password registration and optional Google OAuth.

Backend

  • User model in api/app/models.py: email (unique), hashed password (nullable for OAuth-only), optional oauth_provider/oauth_sub.
  • Auth router at /api/v1/auth/ (api/app/routers/auth.py):
    • POST /register — email + password; returns JWT.
    • POST /login — email + password; returns JWT.
    • GET /me — current user from token.
    • GET /google — redirects to Google consent screen.
    • GET /google/callback — exchanges code, upserts user, redirects to frontend with token in URL hash.
  • Dependencyget_current_user in api/app/deps.py protects all non-auth routes.
  • Tenant isolation: every query in documents, transactions, analytics, anomalies, compare, receipts, export, reports, and chat is filtered by user_id.
  • Per-user upload paths: files are stored under data/raw_docs/user_{id}/.
  • Per-user RAG: FAISS indexes live at data/embeddings/user_{id}/.
  • Chat sessions API at /api/v1/chat/sessions (CRUD scoped to current user).

Frontend

  • AuthProvider in web/src/contexts/auth.tsx: stores JWT in localStorage, exposes login, register, logout, setTokenFromOAuth.
  • Bearer token added to all API requests via web/src/lib/api.ts.
  • Route protection in ShellLayout: unauthenticated users are redirected to /login; public routes: /, /login, /register, /auth/callback.
  • Login/Register pages: Google sign-in is shown only when the API reports OAuth is enabled (GET /auth/config) or NEXT_PUBLIC_GOOGLE_OAUTH_ENABLED=true.
  • Chat sessions sync to the server API when authenticated; fall back to localStorage when offline.

Configuration

VariablePurpose
JWT_SECRETSecret for HS256 token signing (change in production).
JWT_EXPIRE_MINUTESToken validity (default 7 days).
GOOGLE_OAUTH_CLIENT_IDGoogle Cloud console client ID (optional).
GOOGLE_OAUTH_CLIENT_SECRETMatching secret.
GOOGLE_OAUTH_REDIRECT_URIMust match console; default http://localhost:8000/api/v1/auth/google/callback.
FRONTEND_URLWhere the OAuth callback redirects with the token hash fragment.

Public GET /api/v1/auth/config (no auth): returns { "google_oauth_enabled": boolean } so the web UI can hide “Continue with Google” when OAuth is not configured on the server.

Migration

Run python scripts/migrate_database.py from api/ to add user_id columns to existing tables and create users / chat_sessions tables. Existing rows without a user_id are hidden from authenticated queries until backfilled.


Limitations and extension points

AreaLimitationPossible extension
FAISSIndexFlatL2 is linear; slow at very large NIVF / HNSW, or managed vector DB (Pinecone, pgvector, …).
Index updatesadd_documents rebuilds whole indexIncremental add, background jobs, versioning.
Schemacreate_all only; no Alembic in treeMigrations for production schema evolution.
Chathistory trimmed to 20 turns with content (chat.py)Configurable cap, thread storage, or rolling summary.
OCRHost must have Tesseract unless using Docker imageCloud OCR APIs, better layout models.
Compare routesMounted at /documents/... alongside document CRUDEnsure route ordering in OpenAPI matches FastAPI resolution for edge IDs.
SecretsNever commit real GOOGLE_API_KEY; rotate if leakedSecret manager, .env gitignored (already).

This README is the authoritative high-level map of the codebase; for line-level behavior, follow the links into api/app/ and web/src/.

About

AI-powered Intelligent Document Processing (IDP) system with RAG, anomaly detection, and natural language insights. Local, zero-cost alternative to AWS Textract + Bedrock.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

DocSage

DocSage is an agentic document-intelligence system: ingest financial and business documents, persist structured rows (transactions) in a database, build a semantic index for question answering, and expose both through a FastAPI backend and a Next.js UI. The “brain” for generation is Google Gemini; retrieval uses local FAISS + sentence-transformers; multi-step reasoning is orchestrated with LangGraph.


Table of contents


What DocSage does

  1. Upload documents (PDF, images, etc.) via the API or UI.
  2. Parse them through an IDP-style pipeline: text extraction (PDF and/or OCR), heuristic classification, and structured field extraction.
  3. StoreDocument rows plus derived Transaction rows in PostgreSQL or SQLite.
  4. Index in FAISS for RAG: per document, an extraction_summary chunk (from structured extracted_data) when present, plus chunked raw_text—each chunk carries chunk_type, chunk_index, and document metadata for grounding in API responses.
  5. Answer questions via POST /api/v1/chat/insights: a LangGraph workflow can route between fast analytic paths, SQL over documents + transactions (with JOIN on document_id), and full retrieve → rerank → grade → synthesize agentic RAG with HyDE-style query rewriting. Optional history on the request enables multi-turn chat (last 20 turns used server-side); synthesis prompts insist on filename / document_id citations when evidence exists.
flowchart LR
subgraph client [Client]
Browser[Browser_Next.js]
end
subgraph api [FastAPI_api]
Routes[routers]
Agent[LangGraph_agent]
RAG[RAGService_FAISS]
DB[(SQLAlchemy_DB)]
LLM[Gemini_API]
end
Browser -->|HTTP_JSON| Routes
Routes --> Agent
Agent --> RAG
Agent --> DB
Agent --> LLM
Routes --> DB
Loading

Repository map

PathRole
api/Python package app: FastAPI entry api/app/main.py, routers, services, agents, models, api/requirements.txt, api/Dockerfile.
api/app/routers/HTTP route modules (analytics, documents, chat, anomalies, compare, receipts, export, insights report).
api/app/services/Business logic: RAG, LLM wrapper, IDP pipeline, SQL tools, extraction_summary helper (extraction_summary.py), insights, anomalies, export, etc.
api/app/agents/LangGraph graph compiler, nodes, state, langgraph_runner.py.
api/app/vectorstore/faiss_store.py: FAISS + embeddings.
api/scripts/Operational scripts (seed, ingest, embeddings, diagnostics)—not started automatically.
web/Next.js 14 App Router UI; entry layout web/src/app/layout.tsx, pages under web/src/app/.
web/src/lib/api.tsTyped fetch helpers against /api/v1.
scripts/run-dev.shLocal dev: optional Postgres, API + web.
docker-compose.postgres.ymlPostgres-only Compose (used by dev script).
docker-compose.ymlPostgres + API image.
docker-compose.prod.ymlProduction-oriented Compose example.
railway.tomlRailway: Dockerfile build, uvicorn start, /health.
.env.exampleTemplate for api/.env (copy into api/).
data/ (under api/ at runtime)Uploads and indexes: raw_docs, embeddings, processed—paths come from config.py (RAW_DOCS_PATH, FAISS_*, etc.) relative to the API working directory. Do not commit large api/data/ trees; keep them local or on a volume.

Quick start (local)

Prerequisites: Node.js, Python 3.11+, optional Docker for Postgres, Google AI Studio API key for LLM features.

From the repository root:

./scripts/run-dev.sh
  • Uses docker-compose.postgres.yml to start Postgres when Docker is available. If Docker is not running, the API and web still start; use USE_SQLITE=true in api/.env for a file DB, or start Docker and rerun.
  • Copies .env.exampleapi/.env once if missing; ensures web/.env.local has NEXT_PUBLIC_API_URL.
  • Recreates api/.venv if broken; installs Python and npm deps; runs uvicorn and npm run dev.

Ports:API_PORT (default 8000), WEB_PORT (default 3000). UI: http://localhost:3000; OpenAPI: http://localhost:8000/docs.

Busy Postgres port:POSTGRES_PORT=5433 ./scripts/run-dev.sh and set POSTGRES_PORT=5433 in api/.env.

Skip Docker:./scripts/run-dev.sh --no-docker

Database initialization

On API startup, api/app/main.py runs a lifespan hook that calls init_db(): SQLAlchemy create_all for models in api/app/models.py. This creates missing tables (e.g. documents, transactions) but does not run Alembic-style migrations for schema changes.


Configuration

API (api/app/config.py + api/.env)

Settings load from environment and optional api/.env. Unknown keys are ignored (extra="ignore") so stale variables do not crash startup.

VariablePurpose
POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB, POSTGRES_HOST, POSTGRES_PORTPostgreSQL connection when USE_SQLITE is false.
USE_SQLITEIf true, DATABASE_URL is SQLite (sqlite:///./docsage.db).
API_HOST, API_PORTUvicorn bind (used when running app.main as __main__).
GOOGLE_API_KEYGemini API key; alias GEMINI_API_KEY.
GOOGLE_AI_MODELPrimary generateContent model id.
GOOGLE_AI_MODEL_FALLBACKSComma-separated fallback model ids (overload / transient errors / unknown primary).
FAISS_INDEX_PATH, FAISS_DOCUMENTS_PATHPaths to FAISS index file and pickle sidecar for chunk metadata.
EMBEDDING_MODELsentence-transformers model name (default all-MiniLM-L6-v2, 384-d vectors).
RAW_DOCS_PATH, PROCESSED_PATHUpload and processed file roots.
CORS_ORIGINSComma-separated browser origins, or * (dev only; avoid in production).
MAX_UPLOAD_MBUpload size cap for document uploads.
DEBUG, LOG_LEVELApp logging / debug flags.

Web

VariablePurpose
NEXT_PUBLIC_API_URLOrigin of the FastAPI server (no trailing slash), e.g. http://127.0.0.1:8000. Used by web/src/lib/api.ts. Must match the host you use in the browser (localhost vs 127.0.0.1) to avoid CORS/preflight issues.
NEXT_PUBLIC_GOOGLE_OAUTH_ENABLEDOptional. Set to true to show “Continue with Google” before the client fetches GET /api/v1/auth/config; otherwise the UI reads that endpoint and only shows the button when the API has OAuth credentials.
NEXT_PUBLIC_SITE_URLOptional canonical site URL for metadata. If unset on Vercel, VERCEL_URL is used in web/src/app/layout.tsx for metadataBase.

Docker Compose: pass GOOGLE_API_KEY, GOOGLE_AI_MODEL, GOOGLE_AI_MODEL_FALLBACKS into the api service (see docker-compose.yml, docker-compose.prod.yml).


Backend architecture

FastAPI application

api/app/main.py constructs the app with:

  • CORS from settings.cors_origins_list.
  • APIRouter subtree mounted at /api/v1 including analytics, anomalies, documents, compare, receipts, export, insights report, chat.
  • LegacyPOST /chat/insights delegating to the same handler as v1 chat.

Persistence

  • SQLAlchemy 2.x engine + SessionLocal in api/app/db.py.
  • Models in api/app/models.py:
    • Document: filename, path, type, raw text, JSON extracted_data, timestamps.
    • Transaction: document_id, date, amount, vendor, category, description, JSON metadata (ORM attribute meta_data to avoid reserved name issues), confidence / correction flags.
    • DocumentCorrection: audit of user corrections.

Routers use Depends(get_db) for request-scoped sessions.

Validation

api/app/schemas.py defines Pydantic models for HTTP I/O. Notable:

  • QueryRequest: query, use_rag, use_sql, optional history (List[ChatMessage] with role and content). The chat router keeps the last 20 turns with non-empty content. use_rag / use_sql bias LangGraph routing (e.g. SQL-only path when RAG is off and keywords suggest aggregation).

Concept glossary

Each item: what the concept is, then how DocSage applies it (files).

Intelligent Document Processing (IDP)

IDP is the class of systems that turn messy documents (PDFs, scans) into structured, machine-usable data—classification, key-value extraction, validation—not just raw text.

DocSage:api/app/services/idp_pipeline.py implements extraction and heuristic classification; the upload path in api/app/routers/documents.py calls parse_document then persists rows. This is “IDP-inspired”: rules + LLM/heuristics rather than a full enterprise IDP product.

OCR (Optical Character Recognition)

OCR recovers text from pixels (photos, scanned pages).

DocSage:pytesseract with Pillow-compatible inputs in extract_text_with_ocr (idp_pipeline.py). The Docker image installs tesseract-ocr (api/Dockerfile) so containers can OCR without extra host setup.

PDF text extraction

Digital PDFs often expose a text layer; extraction without OCR is faster and more accurate.

DocSage:pdfplumber in extract_text_from_pdf walks pages and concatenates extract_text() output (idp_pipeline.py). Image-only PDFs may still need rasterization + OCR (pipeline-dependent).

Heuristic classification and regex extraction

Heuristics use keywords and patterns to guess document type and pull amounts, dates, and vendors without a dedicated ML model per field.

DocSage:classify_document, extract_amounts, and related helpers in idp_pipeline.py; extracted JSON is stored on Document.extracted_data and Transaction rows are synthesized via extract_transactions_from_document (api/scripts/ingest_docs.py) used from the documents router.

Embeddings

An embedding is a dense vector representing text (or other modalities) in a space where semantic similarityvector proximity.

DocSage:SentenceTransformer in api/app/vectorstore/faiss_store.py encodes strings; default model all-MiniLM-L6-v2 produces 384-dimensional vectors (EMBEDDING_MODEL in config). api/scripts/build_embeddings.py walks all Document rows and, for each, emits (1) a single extraction_summary string via extraction_to_index_text (extraction_summary.py) when extracted_data is present, and (2) sliding-window chunks over raw_text when present. Metadata on each vector row includes chunk_type (extraction_summary vs raw_text), chunk_index, total_chunks (for raw splits), and document id/filename—used downstream for rerank sources and UI citations.

Vector store and approximate search (FAISS)

A vector store indexes vectors for nearest-neighbor search (which chunks are closest to the query embedding).

DocSage:FAISSIndexFlatL2—exact L2 search over all vectors (simple, no training). Index and parallel pickle list of metadata are saved to FAISS_INDEX_PATH / FAISS_DOCUMENTS_PATH (relative to API cwd, typically api/data/embeddings/ in local dev). RAGService loads the index if files exist and exposes search(query, k).

RAG (Retrieval-Augmented Generation)

RAG grounds LLM answers in retrieved passages from a corpus instead of parametric memory alone, reducing hallucination on factual questions about your documents.

DocSage:RAGService wraps the FAISS store; LangGraph nodes call search with a rewritten query after HyDE and optional reranking (api/app/agents/langgraph/nodes.py). node_rerank / node_synthesize attach sources: document_id, filename, chunk_index, chunk_type, score for citation-style UX; final synthesis instructs the model to cite filename and document id when evidence exists and to avoid inventing facts when context and SQL are empty.

LangGraph and agentic control flow

LangGraph models an agent as a state machine: nodes (functions) update state; edges (conditional or fixed) choose the next step. “Agentic” here means multiple LLM and tool steps with branching, not a single prompt.

DocSage:build_agent_graph compiles a StateGraph over AgentState (api/app/agents/langgraph/state.py) including optional history for multi-turn prompts. run_agent_pipeline invokes the compiled graph with graph.invoke, then shapes the response for the REST API.

LangChain packages in api/requirements.txt (langchain-core, langchain-community) support the broader ecosystem; application code under app/ imports LangGraph directly rather than high-level LangChain chains.

HyDE (Hypothetical Document Embeddings)

HyDE asks the LLM to draft a fake answer or passage that would answer the question; that text is embedded and used to retrieve real chunks. It often improves recall when the raw user question is short or mismatched to chunk wording.

DocSage:node_hyde_rewrite in nodes.py calls call_llm to produce a hypothetical block; retrieval uses the rewritten text (see also refinement_hint on failed grades). HyDE, grading, SQL generation, and synthesis prompts also receive a compact conversation block from history when the client sends prior turns.

Retrieval depth, cross-encoder reranking, and grading loop

Two-stage retrieval can mean: (1) cheap bi-encoder over many candidates, then (2) cross-encoder scoring query–passage pairs for a top subset. A grader decides if context is good enough or triggers another retrieval loop.

DocSage: constants RETRIEVE_K, RERANK_POOL, RERANK_KEEP, MAX_RETRIEVAL_LOOPS in nodes.py. cross-encoder/ms-marco-MiniLM-L-6-v2 scores pairs. node_grade sets grade_pass; conditional edges in graph.py send failures back to hyde until the cap, then proceed to optional_sqlsynthesize.

SQL grounding (LLM-generated SELECT)

Grounding here means the LLM sees the real table schema and sample rows before emitting read-only SQL executed against your DB.

DocSage:SQLTools introspects transactions for low-level helpers and exposes get_multitable_sql_llm_context()—combined documents + transactions schemas, truncated raw_text previews, and shrunk extracted_data in document samples (SQLite vs Postgres aware). _generate_sql in nodes.py includes that context, the user question, and a short conversation block from history, and explains that transactions.document_id references documents.id (JOIN allowed). node_sql_only and node_optional_sql merge SQL results into answers; optional SQL is keyword-gated (including document/invoice-style terms).

Fast analytic path (metrics shortcut)

Some questions match precomputed aggregates faster than full RAG.

DocSage:node_route checks keyword hints for vendor/category breakdowns and routes to node_metrics_fast, which calls InsightsService (api/app/services/insights.py) and returns without vector search (nodes.py).

Anomaly detection

Anomaly detection flags unusual rows (duplicates, outliers, date oddities).

DocSage:api/app/services/anomaly_detection.py; exposed via api/app/routers/anomalies.py.

Google Gemini (Generative Language API)

Gemini is accessed through RESTgenerateContent (v1beta), not a proprietary SDK requirement in this repo.

DocSage:api/app/services/llm_service.py builds the request with optional systemInstruction, walks a model chain (primary + GOOGLE_AI_MODEL_FALLBACKS), retries 429 / 5xx with backoff and optional Retry-After, skips to the next model on 400 / 404 (e.g. deprecated model id), and returns assistant text or structured error strings.


LangGraph chat pipeline

Graph topology (mermaid)

Mirrors api/app/agents/langgraph/graph.py.

flowchart TD
entry[route_entry]
route[node_route]
metrics[node_metrics_fast]
sqlOnly[node_sql_only]
hyde[node_hyde_rewrite]
retrieve[node_retrieve]
rerank[node_rerank]
grade[node_grade]
sqlOpt[node_optional_sql]
synth[node_synthesize]
endNode[END]
entry --> route
route -->|metrics_fast| metrics
route -->|sql_only| sqlOnly
route -->|agentic_rag| hyde
hyde --> retrieve
retrieve --> rerank
rerank --> grade
grade -->|retry_HyDE_loop| hyde
grade -->|pass_or_cap| sqlOpt
sqlOpt --> synth
metrics --> endNode
sqlOnly --> endNode
synth --> endNode
Loading

The grade → hyde edge is conditional: only when grade_pass is false and retrieval_iteration is below MAX_RETRIEVAL_LOOPS (_grade_next).

HTTP entry and response shape

api/app/routers/chat.py:

  • Lazily constructs a singleton RAGService (loads FAISS if index files exist).
  • run_chat maps request.history to the graph (last 20 non-empty turns).
  • run_chat calls run_agent_pipeline(query, rag, use_rag=..., use_sql=..., history=...).
  • Returns QueryResponse: answer, sources (list of dicts with document_id, filename, chunk_index, chunk_type, score when RAG ran), sql_query, steps, tool_calls (derived from steps for UI convenience).

When RAG is “skipped” in spirit:use_rag=False with SQL-biased routing yields sql_only. use_rag=False also clears sources in the runner output. use_sql=False disables the optional SQL augmentation node path in the graph state.

Other LLM call sites (outside the graph)

Several features call call_llm directly without LangGraph: e.g. insights report generation (api/app/services/insights_generator.py), categorization (api/app/services/categorization.py), HyDE / synthesize / SQL prompt nodes. The graph is the orchestrator for interactive chat; batch/report flows may be linear.


Document ingestion and data flow

  1. POST/api/v1/documents with multipart file (documents.py).

  2. File bytes saved under RAW_DOCS_PATH.

  3. parse_document(path) runs the IDP pipeline (idp_pipeline.py).

  4. Document inserted; extract_transactions_from_document yields dicts → Transaction rows committed.

  5. FAISS is not automatically rebuilt on every upload. After new imports, extraction changes, or IDP tweaks, refresh vectors so extraction_summary and raw_text chunks stay in sync—for example:

    cd api && ./.venv/bin/python scripts/build_embeddings.py

    You can also use RAGService.build_index / add_documents (rag.py) in custom ops; add_documents currently rebuilds the full index for simplicity.


HTTP API surface

All v1 routes are prefixed with /api/v1 unless noted.

TagMethodPathPurpose
analyticsGET/analytics/summaryDashboard KPIs: counts, spend, averages.
GET/analytics/time-seriesTime-bucketed series for charts.
GET/analytics/vendor-statsTop vendors by spend.
GET/analytics/category-breakdownSpend by category.
GET/analytics/spending-forecastSimple forward-looking projection.
GET/analytics/monthly-spendSpend for a given year/month.
anomaliesGET/anomaliesRule-based anomaly list.
chatPOST/chat/insightsAgentic RAG + SQL pipeline. Body: query (required), use_rag, use_sql, optional history (array of { role, content }, server uses last 20 non-empty turns).
compareGET/documents/{document_id}/similarSimilar documents (e.g. shared vendor / join logic in service).
POST/documents/comparePairwise diff / compare (CompareBody).
documentsGET/documentsList documents (filters, pagination).
POST/documentsUpload + parse + persist transactions.
GET/documents/{id}Metadata.
GET/documents/{id}/detailRich detail payload.
GET/documents/{id}/confidenceExtraction confidence signals.
GET/documents/{id}/previewPreview / annotated stream where implemented.
PATCH/documents/{id}Update extracted JSON (DocumentUpdateBody).
exportsGET/exports/excelDownload Excel export blob.
GET/exports/summaryText/markdown summary for export UX.
insights-reportPOST/insights/generate-reportLLM-generated narrative report.
receipt-matchingGET/receipt-matching/unmatchedQueue of unmatched receipts.
POST/receipt-matching/{receipt_doc_id}/matchLink receipt to candidate transaction.

Legacy (no /api/v1 prefix):POST /chat/insights — same body/response as v1 chat.

Interactive docs: /docs (Swagger UI).


Frontend (web)

Pages (examples):dashboard, chat, documents, insights, anomalies, compare, export, receipt-matching; marketing home composes LandingStory.


Dependencies, scripts, and containers

Python (api/requirements.txt) — grouped by role

GroupExamples
HTTPfastapi, uvicorn, python-multipart, pydantic, pydantic-settings
DBsqlalchemy, psycopg2-binary
Agentslanggraph, langchain-core, langchain-community
Vectors / MLfaiss-cpu, sentence-transformers, numpy, pandas
Documentspdfplumber, pytesseract, Pillow, opencv-python, openpyxl
LLM HTTPrequests

Operational scripts (api/scripts/)

Not invoked by default: seed_db.py, ingest_docs.py, build_embeddings.py, migrate_database.py, add_documents_from_folder.py, diagnose_and_fix_transactions.py, download_huggingface_dataset.py, preload_kaggle_invoices.py, etc. Use them manually for migrations, backfills, demo data, and embedding rebuilds.

Docker

api/DockerfileCOPYs from repo root: docker build -f api/Dockerfile . installs Tesseract system packages, Python deps, copies api/app and api/scripts, creates data/ subtrees, runs uvicorn on port 8000.

Compose files wire Postgres + env; see repository root YAMLs.

Railway

railway.toml: Dockerfile builder, uvicorn app.main:app --host 0.0.0.0 --port $PORT, health check /health, restart policy.


Deployment

  • Frontend: deploy subdirectory web/ (e.g. Vercel). Set NEXT_PUBLIC_API_URL to your API’s public origin. Set NEXT_PUBLIC_SITE_URL or rely on VERCEL_URL for metadata (see layout).
  • Backend: container host (Fly, Railway, Cloud Run, etc.) using api/Dockerfile with root build context. Inject GOOGLE_API_KEY, DB URL, CORS_ORIGINS matching the exact browser origin(s) in production.
  • Persistence: mount a volume (or object storage strategy) for api/data/raw_docs, api/data/embeddings, and the SQLite file if used—ephemeral disks lose indexes and uploads on restart.

Authentication and multi-tenancy

DocSage uses JWT-based authentication with email/password registration and optional Google OAuth.

Backend

  • User model in api/app/models.py: email (unique), hashed password (nullable for OAuth-only), optional oauth_provider/oauth_sub.
  • Auth router at /api/v1/auth/ (api/app/routers/auth.py):
    • POST /register — email + password; returns JWT.
    • POST /login — email + password; returns JWT.
    • GET /me — current user from token.
    • GET /google — redirects to Google consent screen.
    • GET /google/callback — exchanges code, upserts user, redirects to frontend with token in URL hash.
  • Dependencyget_current_user in api/app/deps.py protects all non-auth routes.
  • Tenant isolation: every query in documents, transactions, analytics, anomalies, compare, receipts, export, reports, and chat is filtered by user_id.
  • Per-user upload paths: files are stored under data/raw_docs/user_{id}/.
  • Per-user RAG: FAISS indexes live at data/embeddings/user_{id}/.
  • Chat sessions API at /api/v1/chat/sessions (CRUD scoped to current user).

Frontend

  • AuthProvider in web/src/contexts/auth.tsx: stores JWT in localStorage, exposes login, register, logout, setTokenFromOAuth.
  • Bearer token added to all API requests via web/src/lib/api.ts.
  • Route protection in ShellLayout: unauthenticated users are redirected to /login; public routes: /, /login, /register, /auth/callback.
  • Login/Register pages: Google sign-in is shown only when the API reports OAuth is enabled (GET /auth/config) or NEXT_PUBLIC_GOOGLE_OAUTH_ENABLED=true.
  • Chat sessions sync to the server API when authenticated; fall back to localStorage when offline.

Configuration

VariablePurpose
JWT_SECRETSecret for HS256 token signing (change in production).
JWT_EXPIRE_MINUTESToken validity (default 7 days).
GOOGLE_OAUTH_CLIENT_IDGoogle Cloud console client ID (optional).
GOOGLE_OAUTH_CLIENT_SECRETMatching secret.
GOOGLE_OAUTH_REDIRECT_URIMust match console; default http://localhost:8000/api/v1/auth/google/callback.
FRONTEND_URLWhere the OAuth callback redirects with the token hash fragment.

Public GET /api/v1/auth/config (no auth): returns { "google_oauth_enabled": boolean } so the web UI can hide “Continue with Google” when OAuth is not configured on the server.

Migration

Run python scripts/migrate_database.py from api/ to add user_id columns to existing tables and create users / chat_sessions tables. Existing rows without a user_id are hidden from authenticated queries until backfilled.


Limitations and extension points

AreaLimitationPossible extension
FAISSIndexFlatL2 is linear; slow at very large NIVF / HNSW, or managed vector DB (Pinecone, pgvector, …).
Index updatesadd_documents rebuilds whole indexIncremental add, background jobs, versioning.
Schemacreate_all only; no Alembic in treeMigrations for production schema evolution.
Chathistory trimmed to 20 turns with content (chat.py)Configurable cap, thread storage, or rolling summary.
OCRHost must have Tesseract unless using Docker imageCloud OCR APIs, better layout models.
Compare routesMounted at /documents/... alongside document CRUDEnsure route ordering in OpenAPI matches FastAPI resolution for edge IDs.
SecretsNever commit real GOOGLE_API_KEY; rotate if leakedSecret manager, .env gitignored (already).

This README is the authoritative high-level map of the codebase; for line-level behavior, follow the links into api/app/ and web/src/.

About

AI-powered Intelligent Document Processing (IDP) system with RAG, anomaly detection, and natural language insights. Local, zero-cost alternative to AWS Textract + Bedrock.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

DocSage

DocSage is an agentic document-intelligence system: ingest financial and business documents, persist structured rows (transactions) in a database, build a semantic index for question answering, and expose both through a FastAPI backend and a Next.js UI. The “brain” for generation is Google Gemini; retrieval uses local FAISS + sentence-transformers; multi-step reasoning is orchestrated with LangGraph.


Table of contents


What DocSage does

  1. Upload documents (PDF, images, etc.) via the API or UI.
  2. Parse them through an IDP-style pipeline: text extraction (PDF and/or OCR), heuristic classification, and structured field extraction.
  3. StoreDocument rows plus derived Transaction rows in PostgreSQL or SQLite.
  4. Index in FAISS for RAG: per document, an extraction_summary chunk (from structured extracted_data) when present, plus chunked raw_text—each chunk carries chunk_type, chunk_index, and document metadata for grounding in API responses.
  5. Answer questions via POST /api/v1/chat/insights: a LangGraph workflow can route between fast analytic paths, SQL over documents + transactions (with JOIN on document_id), and full retrieve → rerank → grade → synthesize agentic RAG with HyDE-style query rewriting. Optional history on the request enables multi-turn chat (last 20 turns used server-side); synthesis prompts insist on filename / document_id citations when evidence exists.
flowchart LR
subgraph client [Client]
Browser[Browser_Next.js]
end
subgraph api [FastAPI_api]
Routes[routers]
Agent[LangGraph_agent]
RAG[RAGService_FAISS]
DB[(SQLAlchemy_DB)]
LLM[Gemini_API]
end
Browser -->|HTTP_JSON| Routes
Routes --> Agent
Agent --> RAG
Agent --> DB
Agent --> LLM
Routes --> DB
Loading

Repository map

PathRole
api/Python package app: FastAPI entry api/app/main.py, routers, services, agents, models, api/requirements.txt, api/Dockerfile.
api/app/routers/HTTP route modules (analytics, documents, chat, anomalies, compare, receipts, export, insights report).
api/app/services/Business logic: RAG, LLM wrapper, IDP pipeline, SQL tools, extraction_summary helper (extraction_summary.py), insights, anomalies, export, etc.
api/app/agents/LangGraph graph compiler, nodes, state, langgraph_runner.py.
api/app/vectorstore/faiss_store.py: FAISS + embeddings.
api/scripts/Operational scripts (seed, ingest, embeddings, diagnostics)—not started automatically.
web/Next.js 14 App Router UI; entry layout web/src/app/layout.tsx, pages under web/src/app/.
web/src/lib/api.tsTyped fetch helpers against /api/v1.
scripts/run-dev.shLocal dev: optional Postgres, API + web.
docker-compose.postgres.ymlPostgres-only Compose (used by dev script).
docker-compose.ymlPostgres + API image.
docker-compose.prod.ymlProduction-oriented Compose example.
railway.tomlRailway: Dockerfile build, uvicorn start, /health.
.env.exampleTemplate for api/.env (copy into api/).
data/ (under api/ at runtime)Uploads and indexes: raw_docs, embeddings, processed—paths come from config.py (RAW_DOCS_PATH, FAISS_*, etc.) relative to the API working directory. Do not commit large api/data/ trees; keep them local or on a volume.

Quick start (local)

Prerequisites: Node.js, Python 3.11+, optional Docker for Postgres, Google AI Studio API key for LLM features.

From the repository root:

./scripts/run-dev.sh
  • Uses docker-compose.postgres.yml to start Postgres when Docker is available. If Docker is not running, the API and web still start; use USE_SQLITE=true in api/.env for a file DB, or start Docker and rerun.
  • Copies .env.exampleapi/.env once if missing; ensures web/.env.local has NEXT_PUBLIC_API_URL.
  • Recreates api/.venv if broken; installs Python and npm deps; runs uvicorn and npm run dev.

Ports:API_PORT (default 8000), WEB_PORT (default 3000). UI: http://localhost:3000; OpenAPI: http://localhost:8000/docs.

Busy Postgres port:POSTGRES_PORT=5433 ./scripts/run-dev.sh and set POSTGRES_PORT=5433 in api/.env.

Skip Docker:./scripts/run-dev.sh --no-docker

Database initialization

On API startup, api/app/main.py runs a lifespan hook that calls init_db(): SQLAlchemy create_all for models in api/app/models.py. This creates missing tables (e.g. documents, transactions) but does not run Alembic-style migrations for schema changes.


Configuration

API (api/app/config.py + api/.env)

Settings load from environment and optional api/.env. Unknown keys are ignored (extra="ignore") so stale variables do not crash startup.

VariablePurpose
POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB, POSTGRES_HOST, POSTGRES_PORTPostgreSQL connection when USE_SQLITE is false.
USE_SQLITEIf true, DATABASE_URL is SQLite (sqlite:///./docsage.db).
API_HOST, API_PORTUvicorn bind (used when running app.main as __main__).
GOOGLE_API_KEYGemini API key; alias GEMINI_API_KEY.
GOOGLE_AI_MODELPrimary generateContent model id.
GOOGLE_AI_MODEL_FALLBACKSComma-separated fallback model ids (overload / transient errors / unknown primary).
FAISS_INDEX_PATH, FAISS_DOCUMENTS_PATHPaths to FAISS index file and pickle sidecar for chunk metadata.
EMBEDDING_MODELsentence-transformers model name (default all-MiniLM-L6-v2, 384-d vectors).
RAW_DOCS_PATH, PROCESSED_PATHUpload and processed file roots.
CORS_ORIGINSComma-separated browser origins, or * (dev only; avoid in production).
MAX_UPLOAD_MBUpload size cap for document uploads.
DEBUG, LOG_LEVELApp logging / debug flags.

Web

VariablePurpose
NEXT_PUBLIC_API_URLOrigin of the FastAPI server (no trailing slash), e.g. http://127.0.0.1:8000. Used by web/src/lib/api.ts. Must match the host you use in the browser (localhost vs 127.0.0.1) to avoid CORS/preflight issues.
NEXT_PUBLIC_GOOGLE_OAUTH_ENABLEDOptional. Set to true to show “Continue with Google” before the client fetches GET /api/v1/auth/config; otherwise the UI reads that endpoint and only shows the button when the API has OAuth credentials.
NEXT_PUBLIC_SITE_URLOptional canonical site URL for metadata. If unset on Vercel, VERCEL_URL is used in web/src/app/layout.tsx for metadataBase.

Docker Compose: pass GOOGLE_API_KEY, GOOGLE_AI_MODEL, GOOGLE_AI_MODEL_FALLBACKS into the api service (see docker-compose.yml, docker-compose.prod.yml).


Backend architecture

FastAPI application

api/app/main.py constructs the app with:

  • CORS from settings.cors_origins_list.
  • APIRouter subtree mounted at /api/v1 including analytics, anomalies, documents, compare, receipts, export, insights report, chat.
  • LegacyPOST /chat/insights delegating to the same handler as v1 chat.

Persistence

  • SQLAlchemy 2.x engine + SessionLocal in api/app/db.py.
  • Models in api/app/models.py:
    • Document: filename, path, type, raw text, JSON extracted_data, timestamps.
    • Transaction: document_id, date, amount, vendor, category, description, JSON metadata (ORM attribute meta_data to avoid reserved name issues), confidence / correction flags.
    • DocumentCorrection: audit of user corrections.

Routers use Depends(get_db) for request-scoped sessions.

Validation

api/app/schemas.py defines Pydantic models for HTTP I/O. Notable:

  • QueryRequest: query, use_rag, use_sql, optional history (List[ChatMessage] with role and content). The chat router keeps the last 20 turns with non-empty content. use_rag / use_sql bias LangGraph routing (e.g. SQL-only path when RAG is off and keywords suggest aggregation).

Concept glossary

Each item: what the concept is, then how DocSage applies it (files).

Intelligent Document Processing (IDP)

IDP is the class of systems that turn messy documents (PDFs, scans) into structured, machine-usable data—classification, key-value extraction, validation—not just raw text.

DocSage:api/app/services/idp_pipeline.py implements extraction and heuristic classification; the upload path in api/app/routers/documents.py calls parse_document then persists rows. This is “IDP-inspired”: rules + LLM/heuristics rather than a full enterprise IDP product.

OCR (Optical Character Recognition)

OCR recovers text from pixels (photos, scanned pages).

DocSage:pytesseract with Pillow-compatible inputs in extract_text_with_ocr (idp_pipeline.py). The Docker image installs tesseract-ocr (api/Dockerfile) so containers can OCR without extra host setup.

PDF text extraction

Digital PDFs often expose a text layer; extraction without OCR is faster and more accurate.

DocSage:pdfplumber in extract_text_from_pdf walks pages and concatenates extract_text() output (idp_pipeline.py). Image-only PDFs may still need rasterization + OCR (pipeline-dependent).

Heuristic classification and regex extraction

Heuristics use keywords and patterns to guess document type and pull amounts, dates, and vendors without a dedicated ML model per field.

DocSage:classify_document, extract_amounts, and related helpers in idp_pipeline.py; extracted JSON is stored on Document.extracted_data and Transaction rows are synthesized via extract_transactions_from_document (api/scripts/ingest_docs.py) used from the documents router.

Embeddings

An embedding is a dense vector representing text (or other modalities) in a space where semantic similarityvector proximity.

DocSage:SentenceTransformer in api/app/vectorstore/faiss_store.py encodes strings; default model all-MiniLM-L6-v2 produces 384-dimensional vectors (EMBEDDING_MODEL in config). api/scripts/build_embeddings.py walks all Document rows and, for each, emits (1) a single extraction_summary string via extraction_to_index_text (extraction_summary.py) when extracted_data is present, and (2) sliding-window chunks over raw_text when present. Metadata on each vector row includes chunk_type (extraction_summary vs raw_text), chunk_index, total_chunks (for raw splits), and document id/filename—used downstream for rerank sources and UI citations.

Vector store and approximate search (FAISS)

A vector store indexes vectors for nearest-neighbor search (which chunks are closest to the query embedding).

DocSage:FAISSIndexFlatL2—exact L2 search over all vectors (simple, no training). Index and parallel pickle list of metadata are saved to FAISS_INDEX_PATH / FAISS_DOCUMENTS_PATH (relative to API cwd, typically api/data/embeddings/ in local dev). RAGService loads the index if files exist and exposes search(query, k).

RAG (Retrieval-Augmented Generation)

RAG grounds LLM answers in retrieved passages from a corpus instead of parametric memory alone, reducing hallucination on factual questions about your documents.

DocSage:RAGService wraps the FAISS store; LangGraph nodes call search with a rewritten query after HyDE and optional reranking (api/app/agents/langgraph/nodes.py). node_rerank / node_synthesize attach sources: document_id, filename, chunk_index, chunk_type, score for citation-style UX; final synthesis instructs the model to cite filename and document id when evidence exists and to avoid inventing facts when context and SQL are empty.

LangGraph and agentic control flow

LangGraph models an agent as a state machine: nodes (functions) update state; edges (conditional or fixed) choose the next step. “Agentic” here means multiple LLM and tool steps with branching, not a single prompt.

DocSage:build_agent_graph compiles a StateGraph over AgentState (api/app/agents/langgraph/state.py) including optional history for multi-turn prompts. run_agent_pipeline invokes the compiled graph with graph.invoke, then shapes the response for the REST API.

LangChain packages in api/requirements.txt (langchain-core, langchain-community) support the broader ecosystem; application code under app/ imports LangGraph directly rather than high-level LangChain chains.

HyDE (Hypothetical Document Embeddings)

HyDE asks the LLM to draft a fake answer or passage that would answer the question; that text is embedded and used to retrieve real chunks. It often improves recall when the raw user question is short or mismatched to chunk wording.

DocSage:node_hyde_rewrite in nodes.py calls call_llm to produce a hypothetical block; retrieval uses the rewritten text (see also refinement_hint on failed grades). HyDE, grading, SQL generation, and synthesis prompts also receive a compact conversation block from history when the client sends prior turns.

Retrieval depth, cross-encoder reranking, and grading loop

Two-stage retrieval can mean: (1) cheap bi-encoder over many candidates, then (2) cross-encoder scoring query–passage pairs for a top subset. A grader decides if context is good enough or triggers another retrieval loop.

DocSage: constants RETRIEVE_K, RERANK_POOL, RERANK_KEEP, MAX_RETRIEVAL_LOOPS in nodes.py. cross-encoder/ms-marco-MiniLM-L-6-v2 scores pairs. node_grade sets grade_pass; conditional edges in graph.py send failures back to hyde until the cap, then proceed to optional_sqlsynthesize.

SQL grounding (LLM-generated SELECT)

Grounding here means the LLM sees the real table schema and sample rows before emitting read-only SQL executed against your DB.

DocSage:SQLTools introspects transactions for low-level helpers and exposes get_multitable_sql_llm_context()—combined documents + transactions schemas, truncated raw_text previews, and shrunk extracted_data in document samples (SQLite vs Postgres aware). _generate_sql in nodes.py includes that context, the user question, and a short conversation block from history, and explains that transactions.document_id references documents.id (JOIN allowed). node_sql_only and node_optional_sql merge SQL results into answers; optional SQL is keyword-gated (including document/invoice-style terms).

Fast analytic path (metrics shortcut)

Some questions match precomputed aggregates faster than full RAG.

DocSage:node_route checks keyword hints for vendor/category breakdowns and routes to node_metrics_fast, which calls InsightsService (api/app/services/insights.py) and returns without vector search (nodes.py).

Anomaly detection

Anomaly detection flags unusual rows (duplicates, outliers, date oddities).

DocSage:api/app/services/anomaly_detection.py; exposed via api/app/routers/anomalies.py.

Google Gemini (Generative Language API)

Gemini is accessed through RESTgenerateContent (v1beta), not a proprietary SDK requirement in this repo.

DocSage:api/app/services/llm_service.py builds the request with optional systemInstruction, walks a model chain (primary + GOOGLE_AI_MODEL_FALLBACKS), retries 429 / 5xx with backoff and optional Retry-After, skips to the next model on 400 / 404 (e.g. deprecated model id), and returns assistant text or structured error strings.


LangGraph chat pipeline

Graph topology (mermaid)

Mirrors api/app/agents/langgraph/graph.py.

flowchart TD
entry[route_entry]
route[node_route]
metrics[node_metrics_fast]
sqlOnly[node_sql_only]
hyde[node_hyde_rewrite]
retrieve[node_retrieve]
rerank[node_rerank]
grade[node_grade]
sqlOpt[node_optional_sql]
synth[node_synthesize]
endNode[END]
entry --> route
route -->|metrics_fast| metrics
route -->|sql_only| sqlOnly
route -->|agentic_rag| hyde
hyde --> retrieve
retrieve --> rerank
rerank --> grade
grade -->|retry_HyDE_loop| hyde
grade -->|pass_or_cap| sqlOpt
sqlOpt --> synth
metrics --> endNode
sqlOnly --> endNode
synth --> endNode
Loading

The grade → hyde edge is conditional: only when grade_pass is false and retrieval_iteration is below MAX_RETRIEVAL_LOOPS (_grade_next).

HTTP entry and response shape

api/app/routers/chat.py:

  • Lazily constructs a singleton RAGService (loads FAISS if index files exist).
  • run_chat maps request.history to the graph (last 20 non-empty turns).
  • run_chat calls run_agent_pipeline(query, rag, use_rag=..., use_sql=..., history=...).
  • Returns QueryResponse: answer, sources (list of dicts with document_id, filename, chunk_index, chunk_type, score when RAG ran), sql_query, steps, tool_calls (derived from steps for UI convenience).

When RAG is “skipped” in spirit:use_rag=False with SQL-biased routing yields sql_only. use_rag=False also clears sources in the runner output. use_sql=False disables the optional SQL augmentation node path in the graph state.

Other LLM call sites (outside the graph)

Several features call call_llm directly without LangGraph: e.g. insights report generation (api/app/services/insights_generator.py), categorization (api/app/services/categorization.py), HyDE / synthesize / SQL prompt nodes. The graph is the orchestrator for interactive chat; batch/report flows may be linear.


Document ingestion and data flow

  1. POST/api/v1/documents with multipart file (documents.py).

  2. File bytes saved under RAW_DOCS_PATH.

  3. parse_document(path) runs the IDP pipeline (idp_pipeline.py).

  4. Document inserted; extract_transactions_from_document yields dicts → Transaction rows committed.

  5. FAISS is not automatically rebuilt on every upload. After new imports, extraction changes, or IDP tweaks, refresh vectors so extraction_summary and raw_text chunks stay in sync—for example:

    cd api && ./.venv/bin/python scripts/build_embeddings.py

    You can also use RAGService.build_index / add_documents (rag.py) in custom ops; add_documents currently rebuilds the full index for simplicity.


HTTP API surface

All v1 routes are prefixed with /api/v1 unless noted.

TagMethodPathPurpose
analyticsGET/analytics/summaryDashboard KPIs: counts, spend, averages.
GET/analytics/time-seriesTime-bucketed series for charts.
GET/analytics/vendor-statsTop vendors by spend.
GET/analytics/category-breakdownSpend by category.
GET/analytics/spending-forecastSimple forward-looking projection.
GET/analytics/monthly-spendSpend for a given year/month.
anomaliesGET/anomaliesRule-based anomaly list.
chatPOST/chat/insightsAgentic RAG + SQL pipeline. Body: query (required), use_rag, use_sql, optional history (array of { role, content }, server uses last 20 non-empty turns).
compareGET/documents/{document_id}/similarSimilar documents (e.g. shared vendor / join logic in service).
POST/documents/comparePairwise diff / compare (CompareBody).
documentsGET/documentsList documents (filters, pagination).
POST/documentsUpload + parse + persist transactions.
GET/documents/{id}Metadata.
GET/documents/{id}/detailRich detail payload.
GET/documents/{id}/confidenceExtraction confidence signals.
GET/documents/{id}/previewPreview / annotated stream where implemented.
PATCH/documents/{id}Update extracted JSON (DocumentUpdateBody).
exportsGET/exports/excelDownload Excel export blob.
GET/exports/summaryText/markdown summary for export UX.
insights-reportPOST/insights/generate-reportLLM-generated narrative report.
receipt-matchingGET/receipt-matching/unmatchedQueue of unmatched receipts.
POST/receipt-matching/{receipt_doc_id}/matchLink receipt to candidate transaction.

Legacy (no /api/v1 prefix):POST /chat/insights — same body/response as v1 chat.

Interactive docs: /docs (Swagger UI).


Frontend (web)

Pages (examples):dashboard, chat, documents, insights, anomalies, compare, export, receipt-matching; marketing home composes LandingStory.


Dependencies, scripts, and containers

Python (api/requirements.txt) — grouped by role

GroupExamples
HTTPfastapi, uvicorn, python-multipart, pydantic, pydantic-settings
DBsqlalchemy, psycopg2-binary
Agentslanggraph, langchain-core, langchain-community
Vectors / MLfaiss-cpu, sentence-transformers, numpy, pandas
Documentspdfplumber, pytesseract, Pillow, opencv-python, openpyxl
LLM HTTPrequests

Operational scripts (api/scripts/)

Not invoked by default: seed_db.py, ingest_docs.py, build_embeddings.py, migrate_database.py, add_documents_from_folder.py, diagnose_and_fix_transactions.py, download_huggingface_dataset.py, preload_kaggle_invoices.py, etc. Use them manually for migrations, backfills, demo data, and embedding rebuilds.

Docker

api/DockerfileCOPYs from repo root: docker build -f api/Dockerfile . installs Tesseract system packages, Python deps, copies api/app and api/scripts, creates data/ subtrees, runs uvicorn on port 8000.

Compose files wire Postgres + env; see repository root YAMLs.

Railway

railway.toml: Dockerfile builder, uvicorn app.main:app --host 0.0.0.0 --port $PORT, health check /health, restart policy.


Deployment

  • Frontend: deploy subdirectory web/ (e.g. Vercel). Set NEXT_PUBLIC_API_URL to your API’s public origin. Set NEXT_PUBLIC_SITE_URL or rely on VERCEL_URL for metadata (see layout).
  • Backend: container host (Fly, Railway, Cloud Run, etc.) using api/Dockerfile with root build context. Inject GOOGLE_API_KEY, DB URL, CORS_ORIGINS matching the exact browser origin(s) in production.
  • Persistence: mount a volume (or object storage strategy) for api/data/raw_docs, api/data/embeddings, and the SQLite file if used—ephemeral disks lose indexes and uploads on restart.

Authentication and multi-tenancy

DocSage uses JWT-based authentication with email/password registration and optional Google OAuth.

Backend

  • User model in api/app/models.py: email (unique), hashed password (nullable for OAuth-only), optional oauth_provider/oauth_sub.
  • Auth router at /api/v1/auth/ (api/app/routers/auth.py):
    • POST /register — email + password; returns JWT.
    • POST /login — email + password; returns JWT.
    • GET /me — current user from token.
    • GET /google — redirects to Google consent screen.
    • GET /google/callback — exchanges code, upserts user, redirects to frontend with token in URL hash.
  • Dependencyget_current_user in api/app/deps.py protects all non-auth routes.
  • Tenant isolation: every query in documents, transactions, analytics, anomalies, compare, receipts, export, reports, and chat is filtered by user_id.
  • Per-user upload paths: files are stored under data/raw_docs/user_{id}/.
  • Per-user RAG: FAISS indexes live at data/embeddings/user_{id}/.
  • Chat sessions API at /api/v1/chat/sessions (CRUD scoped to current user).

Frontend

  • AuthProvider in web/src/contexts/auth.tsx: stores JWT in localStorage, exposes login, register, logout, setTokenFromOAuth.
  • Bearer token added to all API requests via web/src/lib/api.ts.
  • Route protection in ShellLayout: unauthenticated users are redirected to /login; public routes: /, /login, /register, /auth/callback.
  • Login/Register pages: Google sign-in is shown only when the API reports OAuth is enabled (GET /auth/config) or NEXT_PUBLIC_GOOGLE_OAUTH_ENABLED=true.
  • Chat sessions sync to the server API when authenticated; fall back to localStorage when offline.

Configuration

VariablePurpose
JWT_SECRETSecret for HS256 token signing (change in production).
JWT_EXPIRE_MINUTESToken validity (default 7 days).
GOOGLE_OAUTH_CLIENT_IDGoogle Cloud console client ID (optional).
GOOGLE_OAUTH_CLIENT_SECRETMatching secret.
GOOGLE_OAUTH_REDIRECT_URIMust match console; default http://localhost:8000/api/v1/auth/google/callback.
FRONTEND_URLWhere the OAuth callback redirects with the token hash fragment.

Public GET /api/v1/auth/config (no auth): returns { "google_oauth_enabled": boolean } so the web UI can hide “Continue with Google” when OAuth is not configured on the server.

Migration

Run python scripts/migrate_database.py from api/ to add user_id columns to existing tables and create users / chat_sessions tables. Existing rows without a user_id are hidden from authenticated queries until backfilled.


Limitations and extension points

AreaLimitationPossible extension
FAISSIndexFlatL2 is linear; slow at very large NIVF / HNSW, or managed vector DB (Pinecone, pgvector, …).
Index updatesadd_documents rebuilds whole indexIncremental add, background jobs, versioning.
Schemacreate_all only; no Alembic in treeMigrations for production schema evolution.
Chathistory trimmed to 20 turns with content (chat.py)Configurable cap, thread storage, or rolling summary.
OCRHost must have Tesseract unless using Docker imageCloud OCR APIs, better layout models.
Compare routesMounted at /documents/... alongside document CRUDEnsure route ordering in OpenAPI matches FastAPI resolution for edge IDs.
SecretsNever commit real GOOGLE_API_KEY; rotate if leakedSecret manager, .env gitignored (already).

This README is the authoritative high-level map of the codebase; for line-level behavior, follow the links into api/app/ and web/src/.

About

AI-powered Intelligent Document Processing (IDP) system with RAG, anomaly detection, and natural language insights. Local, zero-cost alternative to AWS Textract + Bedrock.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Repository files navigation

DocSage

DocSage is an agentic document-intelligence system: ingest financial and business documents, persist structured rows (transactions) in a database, build a semantic index for question answering, and expose both through a FastAPI backend and a Next.js UI. The “brain” for generation is Google Gemini; retrieval uses local FAISS + sentence-transformers; multi-step reasoning is orchestrated with LangGraph.


Table of contents


What DocSage does

  1. Upload documents (PDF, images, etc.) via the API or UI.
  2. Parse them through an IDP-style pipeline: text extraction (PDF and/or OCR), heuristic classification, and structured field extraction.
  3. StoreDocument rows plus derived Transaction rows in PostgreSQL or SQLite.
  4. Index in FAISS for RAG: per document, an extraction_summary chunk (from structured extracted_data) when present, plus chunked raw_text—each chunk carries chunk_type, chunk_index, and document metadata for grounding in API responses.
  5. Answer questions via POST /api/v1/chat/insights: a LangGraph workflow can route between fast analytic paths, SQL over documents + transactions (with JOIN on document_id), and full retrieve → rerank → grade → synthesize agentic RAG with HyDE-style query rewriting. Optional history on the request enables multi-turn chat (last 20 turns used server-side); synthesis prompts insist on filename / document_id citations when evidence exists.
flowchart LR
subgraph client [Client]
Browser[Browser_Next.js]
end
subgraph api [FastAPI_api]
Routes[routers]
Agent[LangGraph_agent]
RAG[RAGService_FAISS]
DB[(SQLAlchemy_DB)]
LLM[Gemini_API]
end
Browser -->|HTTP_JSON| Routes
Routes --> Agent
Agent --> RAG
Agent --> DB
Agent --> LLM
Routes --> DB
Loading

Repository map

PathRole
api/Python package app: FastAPI entry api/app/main.py, routers, services, agents, models, api/requirements.txt, api/Dockerfile.
api/app/routers/HTTP route modules (analytics, documents, chat, anomalies, compare, receipts, export, insights report).
api/app/services/Business logic: RAG, LLM wrapper, IDP pipeline, SQL tools, extraction_summary helper (extraction_summary.py), insights, anomalies, export, etc.
api/app/agents/LangGraph graph compiler, nodes, state, langgraph_runner.py.
api/app/vectorstore/faiss_store.py: FAISS + embeddings.
api/scripts/Operational scripts (seed, ingest, embeddings, diagnostics)—not started automatically.
web/Next.js 14 App Router UI; entry layout web/src/app/layout.tsx, pages under web/src/app/.
web/src/lib/api.tsTyped fetch helpers against /api/v1.
scripts/run-dev.shLocal dev: optional Postgres, API + web.
docker-compose.postgres.ymlPostgres-only Compose (used by dev script).
docker-compose.ymlPostgres + API image.
docker-compose.prod.ymlProduction-oriented Compose example.
railway.tomlRailway: Dockerfile build, uvicorn start, /health.
.env.exampleTemplate for api/.env (copy into api/).
data/ (under api/ at runtime)Uploads and indexes: raw_docs, embeddings, processed—paths come from config.py (RAW_DOCS_PATH, FAISS_*, etc.) relative to the API working directory. Do not commit large api/data/ trees; keep them local or on a volume.

Quick start (local)

Prerequisites: Node.js, Python 3.11+, optional Docker for Postgres, Google AI Studio API key for LLM features.

From the repository root:

./scripts/run-dev.sh
  • Uses docker-compose.postgres.yml to start Postgres when Docker is available. If Docker is not running, the API and web still start; use USE_SQLITE=true in api/.env for a file DB, or start Docker and rerun.
  • Copies .env.exampleapi/.env once if missing; ensures web/.env.local has NEXT_PUBLIC_API_URL.
  • Recreates api/.venv if broken; installs Python and npm deps; runs uvicorn and npm run dev.

Ports:API_PORT (default 8000), WEB_PORT (default 3000). UI: http://localhost:3000; OpenAPI: http://localhost:8000/docs.

Busy Postgres port:POSTGRES_PORT=5433 ./scripts/run-dev.sh and set POSTGRES_PORT=5433 in api/.env.

Skip Docker:./scripts/run-dev.sh --no-docker

Database initialization

On API startup, api/app/main.py runs a lifespan hook that calls init_db(): SQLAlchemy create_all for models in api/app/models.py. This creates missing tables (e.g. documents, transactions) but does not run Alembic-style migrations for schema changes.


Configuration

API (api/app/config.py + api/.env)

Settings load from environment and optional api/.env. Unknown keys are ignored (extra="ignore") so stale variables do not crash startup.

VariablePurpose
POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB, POSTGRES_HOST, POSTGRES_PORTPostgreSQL connection when USE_SQLITE is false.
USE_SQLITEIf true, DATABASE_URL is SQLite (sqlite:///./docsage.db).
API_HOST, API_PORTUvicorn bind (used when running app.main as __main__).
GOOGLE_API_KEYGemini API key; alias GEMINI_API_KEY.
GOOGLE_AI_MODELPrimary generateContent model id.
GOOGLE_AI_MODEL_FALLBACKSComma-separated fallback model ids (overload / transient errors / unknown primary).
FAISS_INDEX_PATH, FAISS_DOCUMENTS_PATHPaths to FAISS index file and pickle sidecar for chunk metadata.
EMBEDDING_MODELsentence-transformers model name (default all-MiniLM-L6-v2, 384-d vectors).
RAW_DOCS_PATH, PROCESSED_PATHUpload and processed file roots.
CORS_ORIGINSComma-separated browser origins, or * (dev only; avoid in production).
MAX_UPLOAD_MBUpload size cap for document uploads.
DEBUG, LOG_LEVELApp logging / debug flags.

Web

VariablePurpose
NEXT_PUBLIC_API_URLOrigin of the FastAPI server (no trailing slash), e.g. http://127.0.0.1:8000. Used by web/src/lib/api.ts. Must match the host you use in the browser (localhost vs 127.0.0.1) to avoid CORS/preflight issues.
NEXT_PUBLIC_GOOGLE_OAUTH_ENABLEDOptional. Set to true to show “Continue with Google” before the client fetches GET /api/v1/auth/config; otherwise the UI reads that endpoint and only shows the button when the API has OAuth credentials.
NEXT_PUBLIC_SITE_URLOptional canonical site URL for metadata. If unset on Vercel, VERCEL_URL is used in web/src/app/layout.tsx for metadataBase.

Docker Compose: pass GOOGLE_API_KEY, GOOGLE_AI_MODEL, GOOGLE_AI_MODEL_FALLBACKS into the api service (see docker-compose.yml, docker-compose.prod.yml).


Backend architecture

FastAPI application

api/app/main.py constructs the app with:

  • CORS from settings.cors_origins_list.
  • APIRouter subtree mounted at /api/v1 including analytics, anomalies, documents, compare, receipts, export, insights report, chat.
  • LegacyPOST /chat/insights delegating to the same handler as v1 chat.

Persistence

  • SQLAlchemy 2.x engine + SessionLocal in api/app/db.py.
  • Models in api/app/models.py:
    • Document: filename, path, type, raw text, JSON extracted_data, timestamps.
    • Transaction: document_id, date, amount, vendor, category, description, JSON metadata (ORM attribute meta_data to avoid reserved name issues), confidence / correction flags.
    • DocumentCorrection: audit of user corrections.

Routers use Depends(get_db) for request-scoped sessions.

Validation

api/app/schemas.py defines Pydantic models for HTTP I/O. Notable:

  • QueryRequest: query, use_rag, use_sql, optional history (List[ChatMessage] with role and content). The chat router keeps the last 20 turns with non-empty content. use_rag / use_sql bias LangGraph routing (e.g. SQL-only path when RAG is off and keywords suggest aggregation).

Concept glossary

Each item: what the concept is, then how DocSage applies it (files).

Intelligent Document Processing (IDP)

IDP is the class of systems that turn messy documents (PDFs, scans) into structured, machine-usable data—classification, key-value extraction, validation—not just raw text.

DocSage:api/app/services/idp_pipeline.py implements extraction and heuristic classification; the upload path in api/app/routers/documents.py calls parse_document then persists rows. This is “IDP-inspired”: rules + LLM/heuristics rather than a full enterprise IDP product.

OCR (Optical Character Recognition)

OCR recovers text from pixels (photos, scanned pages).

DocSage:pytesseract with Pillow-compatible inputs in extract_text_with_ocr (idp_pipeline.py). The Docker image installs tesseract-ocr (api/Dockerfile) so containers can OCR without extra host setup.

PDF text extraction

Digital PDFs often expose a text layer; extraction without OCR is faster and more accurate.

DocSage:pdfplumber in extract_text_from_pdf walks pages and concatenates extract_text() output (idp_pipeline.py). Image-only PDFs may still need rasterization + OCR (pipeline-dependent).

Heuristic classification and regex extraction

Heuristics use keywords and patterns to guess document type and pull amounts, dates, and vendors without a dedicated ML model per field.

DocSage:classify_document, extract_amounts, and related helpers in idp_pipeline.py; extracted JSON is stored on Document.extracted_data and Transaction rows are synthesized via extract_transactions_from_document (api/scripts/ingest_docs.py) used from the documents router.

Embeddings

An embedding is a dense vector representing text (or other modalities) in a space where semantic similarityvector proximity.

DocSage:SentenceTransformer in api/app/vectorstore/faiss_store.py encodes strings; default model all-MiniLM-L6-v2 produces 384-dimensional vectors (EMBEDDING_MODEL in config). api/scripts/build_embeddings.py walks all Document rows and, for each, emits (1) a single extraction_summary string via extraction_to_index_text (extraction_summary.py) when extracted_data is present, and (2) sliding-window chunks over raw_text when present. Metadata on each vector row includes chunk_type (extraction_summary vs raw_text), chunk_index, total_chunks (for raw splits), and document id/filename—used downstream for rerank sources and UI citations.

Vector store and approximate search (FAISS)

A vector store indexes vectors for nearest-neighbor search (which chunks are closest to the query embedding).

DocSage:FAISSIndexFlatL2—exact L2 search over all vectors (simple, no training). Index and parallel pickle list of metadata are saved to FAISS_INDEX_PATH / FAISS_DOCUMENTS_PATH (relative to API cwd, typically api/data/embeddings/ in local dev). RAGService loads the index if files exist and exposes search(query, k).

RAG (Retrieval-Augmented Generation)

RAG grounds LLM answers in retrieved passages from a corpus instead of parametric memory alone, reducing hallucination on factual questions about your documents.

DocSage:RAGService wraps the FAISS store; LangGraph nodes call search with a rewritten query after HyDE and optional reranking (api/app/agents/langgraph/nodes.py). node_rerank / node_synthesize attach sources: document_id, filename, chunk_index, chunk_type, score for citation-style UX; final synthesis instructs the model to cite filename and document id when evidence exists and to avoid inventing facts when context and SQL are empty.

LangGraph and agentic control flow

LangGraph models an agent as a state machine: nodes (functions) update state; edges (conditional or fixed) choose the next step. “Agentic” here means multiple LLM and tool steps with branching, not a single prompt.

DocSage:build_agent_graph compiles a StateGraph over AgentState (api/app/agents/langgraph/state.py) including optional history for multi-turn prompts. run_agent_pipeline invokes the compiled graph with graph.invoke, then shapes the response for the REST API.

LangChain packages in api/requirements.txt (langchain-core, langchain-community) support the broader ecosystem; application code under app/ imports LangGraph directly rather than high-level LangChain chains.

HyDE (Hypothetical Document Embeddings)

HyDE asks the LLM to draft a fake answer or passage that would answer the question; that text is embedded and used to retrieve real chunks. It often improves recall when the raw user question is short or mismatched to chunk wording.

DocSage:node_hyde_rewrite in nodes.py calls call_llm to produce a hypothetical block; retrieval uses the rewritten text (see also refinement_hint on failed grades). HyDE, grading, SQL generation, and synthesis prompts also receive a compact conversation block from history when the client sends prior turns.

Retrieval depth, cross-encoder reranking, and grading loop

Two-stage retrieval can mean: (1) cheap bi-encoder over many candidates, then (2) cross-encoder scoring query–passage pairs for a top subset. A grader decides if context is good enough or triggers another retrieval loop.

DocSage: constants RETRIEVE_K, RERANK_POOL, RERANK_KEEP, MAX_RETRIEVAL_LOOPS in nodes.py. cross-encoder/ms-marco-MiniLM-L-6-v2 scores pairs. node_grade sets grade_pass; conditional edges in graph.py send failures back to hyde until the cap, then proceed to optional_sqlsynthesize.

SQL grounding (LLM-generated SELECT)

Grounding here means the LLM sees the real table schema and sample rows before emitting read-only SQL executed against your DB.

DocSage:SQLTools introspects transactions for low-level helpers and exposes get_multitable_sql_llm_context()—combined documents + transactions schemas, truncated raw_text previews, and shrunk extracted_data in document samples (SQLite vs Postgres aware). _generate_sql in nodes.py includes that context, the user question, and a short conversation block from history, and explains that transactions.document_id references documents.id (JOIN allowed). node_sql_only and node_optional_sql merge SQL results into answers; optional SQL is keyword-gated (including document/invoice-style terms).

Fast analytic path (metrics shortcut)

Some questions match precomputed aggregates faster than full RAG.

DocSage:node_route checks keyword hints for vendor/category breakdowns and routes to node_metrics_fast, which calls InsightsService (api/app/services/insights.py) and returns without vector search (nodes.py).

Anomaly detection

Anomaly detection flags unusual rows (duplicates, outliers, date oddities).

DocSage:api/app/services/anomaly_detection.py; exposed via api/app/routers/anomalies.py.

Google Gemini (Generative Language API)

Gemini is accessed through RESTgenerateContent (v1beta), not a proprietary SDK requirement in this repo.

DocSage:api/app/services/llm_service.py builds the request with optional systemInstruction, walks a model chain (primary + GOOGLE_AI_MODEL_FALLBACKS), retries 429 / 5xx with backoff and optional Retry-After, skips to the next model on 400 / 404 (e.g. deprecated model id), and returns assistant text or structured error strings.


LangGraph chat pipeline

Graph topology (mermaid)

Mirrors api/app/agents/langgraph/graph.py.

flowchart TD
entry[route_entry]
route[node_route]
metrics[node_metrics_fast]
sqlOnly[node_sql_only]
hyde[node_hyde_rewrite]
retrieve[node_retrieve]
rerank[node_rerank]
grade[node_grade]
sqlOpt[node_optional_sql]
synth[node_synthesize]
endNode[END]
entry --> route
route -->|metrics_fast| metrics
route -->|sql_only| sqlOnly
route -->|agentic_rag| hyde
hyde --> retrieve
retrieve --> rerank
rerank --> grade
grade -->|retry_HyDE_loop| hyde
grade -->|pass_or_cap| sqlOpt
sqlOpt --> synth
metrics --> endNode
sqlOnly --> endNode
synth --> endNode
Loading

The grade → hyde edge is conditional: only when grade_pass is false and retrieval_iteration is below MAX_RETRIEVAL_LOOPS (_grade_next).

HTTP entry and response shape

api/app/routers/chat.py:

  • Lazily constructs a singleton RAGService (loads FAISS if index files exist).
  • run_chat maps request.history to the graph (last 20 non-empty turns).
  • run_chat calls run_agent_pipeline(query, rag, use_rag=..., use_sql=..., history=...).
  • Returns QueryResponse: answer, sources (list of dicts with document_id, filename, chunk_index, chunk_type, score when RAG ran), sql_query, steps, tool_calls (derived from steps for UI convenience).

When RAG is “skipped” in spirit:use_rag=False with SQL-biased routing yields sql_only. use_rag=False also clears sources in the runner output. use_sql=False disables the optional SQL augmentation node path in the graph state.

Other LLM call sites (outside the graph)

Several features call call_llm directly without LangGraph: e.g. insights report generation (api/app/services/insights_generator.py), categorization (api/app/services/categorization.py), HyDE / synthesize / SQL prompt nodes. The graph is the orchestrator for interactive chat; batch/report flows may be linear.


Document ingestion and data flow

  1. POST/api/v1/documents with multipart file (documents.py).

  2. File bytes saved under RAW_DOCS_PATH.

  3. parse_document(path) runs the IDP pipeline (idp_pipeline.py).

  4. Document inserted; extract_transactions_from_document yields dicts → Transaction rows committed.

  5. FAISS is not automatically rebuilt on every upload. After new imports, extraction changes, or IDP tweaks, refresh vectors so extraction_summary and raw_text chunks stay in sync—for example:

    cd api && ./.venv/bin/python scripts/build_embeddings.py

    You can also use RAGService.build_index / add_documents (rag.py) in custom ops; add_documents currently rebuilds the full index for simplicity.


HTTP API surface

All v1 routes are prefixed with /api/v1 unless noted.

TagMethodPathPurpose
analyticsGET/analytics/summaryDashboard KPIs: counts, spend, averages.
GET/analytics/time-seriesTime-bucketed series for charts.
GET/analytics/vendor-statsTop vendors by spend.
GET/analytics/category-breakdownSpend by category.
GET/analytics/spending-forecastSimple forward-looking projection.
GET/analytics/monthly-spendSpend for a given year/month.
anomaliesGET/anomaliesRule-based anomaly list.
chatPOST/chat/insightsAgentic RAG + SQL pipeline. Body: query (required), use_rag, use_sql, optional history (array of { role, content }, server uses last 20 non-empty turns).
compareGET/documents/{document_id}/similarSimilar documents (e.g. shared vendor / join logic in service).
POST/documents/comparePairwise diff / compare (CompareBody).
documentsGET/documentsList documents (filters, pagination).
POST/documentsUpload + parse + persist transactions.
GET/documents/{id}Metadata.
GET/documents/{id}/detailRich detail payload.
GET/documents/{id}/confidenceExtraction confidence signals.
GET/documents/{id}/previewPreview / annotated stream where implemented.
PATCH/documents/{id}Update extracted JSON (DocumentUpdateBody).
exportsGET/exports/excelDownload Excel export blob.
GET/exports/summaryText/markdown summary for export UX.
insights-reportPOST/insights/generate-reportLLM-generated narrative report.
receipt-matchingGET/receipt-matching/unmatchedQueue of unmatched receipts.
POST/receipt-matching/{receipt_doc_id}/matchLink receipt to candidate transaction.

Legacy (no /api/v1 prefix):POST /chat/insights — same body/response as v1 chat.

Interactive docs: /docs (Swagger UI).


Frontend (web)

Pages (examples):dashboard, chat, documents, insights, anomalies, compare, export, receipt-matching; marketing home composes LandingStory.


Dependencies, scripts, and containers

Python (api/requirements.txt) — grouped by role

GroupExamples
HTTPfastapi, uvicorn, python-multipart, pydantic, pydantic-settings
DBsqlalchemy, psycopg2-binary
Agentslanggraph, langchain-core, langchain-community
Vectors / MLfaiss-cpu, sentence-transformers, numpy, pandas
Documentspdfplumber, pytesseract, Pillow, opencv-python, openpyxl
LLM HTTPrequests

Operational scripts (api/scripts/)

Not invoked by default: seed_db.py, ingest_docs.py, build_embeddings.py, migrate_database.py, add_documents_from_folder.py, diagnose_and_fix_transactions.py, download_huggingface_dataset.py, preload_kaggle_invoices.py, etc. Use them manually for migrations, backfills, demo data, and embedding rebuilds.

Docker

api/DockerfileCOPYs from repo root: docker build -f api/Dockerfile . installs Tesseract system packages, Python deps, copies api/app and api/scripts, creates data/ subtrees, runs uvicorn on port 8000.

Compose files wire Postgres + env; see repository root YAMLs.

Railway

railway.toml: Dockerfile builder, uvicorn app.main:app --host 0.0.0.0 --port $PORT, health check /health, restart policy.


Deployment

  • Frontend: deploy subdirectory web/ (e.g. Vercel). Set NEXT_PUBLIC_API_URL to your API’s public origin. Set NEXT_PUBLIC_SITE_URL or rely on VERCEL_URL for metadata (see layout).
  • Backend: container host (Fly, Railway, Cloud Run, etc.) using api/Dockerfile with root build context. Inject GOOGLE_API_KEY, DB URL, CORS_ORIGINS matching the exact browser origin(s) in production.
  • Persistence: mount a volume (or object storage strategy) for api/data/raw_docs, api/data/embeddings, and the SQLite file if used—ephemeral disks lose indexes and uploads on restart.

Authentication and multi-tenancy

DocSage uses JWT-based authentication with email/password registration and optional Google OAuth.

Backend

  • User model in api/app/models.py: email (unique), hashed password (nullable for OAuth-only), optional oauth_provider/oauth_sub.
  • Auth router at /api/v1/auth/ (api/app/routers/auth.py):
    • POST /register — email + password; returns JWT.
    • POST /login — email + password; returns JWT.
    • GET /me — current user from token.
    • GET /google — redirects to Google consent screen.
    • GET /google/callback — exchanges code, upserts user, redirects to frontend with token in URL hash.
  • Dependencyget_current_user in api/app/deps.py protects all non-auth routes.
  • Tenant isolation: every query in documents, transactions, analytics, anomalies, compare, receipts, export, reports, and chat is filtered by user_id.
  • Per-user upload paths: files are stored under data/raw_docs/user_{id}/.
  • Per-user RAG: FAISS indexes live at data/embeddings/user_{id}/.
  • Chat sessions API at /api/v1/chat/sessions (CRUD scoped to current user).

Frontend

  • AuthProvider in web/src/contexts/auth.tsx: stores JWT in localStorage, exposes login, register, logout, setTokenFromOAuth.
  • Bearer token added to all API requests via web/src/lib/api.ts.
  • Route protection in ShellLayout: unauthenticated users are redirected to /login; public routes: /, /login, /register, /auth/callback.
  • Login/Register pages: Google sign-in is shown only when the API reports OAuth is enabled (GET /auth/config) or NEXT_PUBLIC_GOOGLE_OAUTH_ENABLED=true.
  • Chat sessions sync to the server API when authenticated; fall back to localStorage when offline.

Configuration

VariablePurpose
JWT_SECRETSecret for HS256 token signing (change in production).
JWT_EXPIRE_MINUTESToken validity (default 7 days).
GOOGLE_OAUTH_CLIENT_IDGoogle Cloud console client ID (optional).
GOOGLE_OAUTH_CLIENT_SECRETMatching secret.
GOOGLE_OAUTH_REDIRECT_URIMust match console; default http://localhost:8000/api/v1/auth/google/callback.
FRONTEND_URLWhere the OAuth callback redirects with the token hash fragment.

Public GET /api/v1/auth/config (no auth): returns { "google_oauth_enabled": boolean } so the web UI can hide “Continue with Google” when OAuth is not configured on the server.

Migration

Run python scripts/migrate_database.py from api/ to add user_id columns to existing tables and create users / chat_sessions tables. Existing rows without a user_id are hidden from authenticated queries until backfilled.


Limitations and extension points

AreaLimitationPossible extension
FAISSIndexFlatL2 is linear; slow at very large NIVF / HNSW, or managed vector DB (Pinecone, pgvector, …).
Index updatesadd_documents rebuilds whole indexIncremental add, background jobs, versioning.
Schemacreate_all only; no Alembic in treeMigrations for production schema evolution.
Chathistory trimmed to 20 turns with content (chat.py)Configurable cap, thread storage, or rolling summary.
OCRHost must have Tesseract unless using Docker imageCloud OCR APIs, better layout models.
Compare routesMounted at /documents/... alongside document CRUDEnsure route ordering in OpenAPI matches FastAPI resolution for edge IDs.
SecretsNever commit real GOOGLE_API_KEY; rotate if leakedSecret manager, .env gitignored (already).

This README is the authoritative high-level map of the codebase; for line-level behavior, follow the links into api/app/ and web/src/.

About

AI-powered Intelligent Document Processing (IDP) system with RAG, anomaly detection, and natural language insights. Local, zero-cost alternative to AWS Textract + Bedrock.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

DocSage

DocSage is an agentic document-intelligence system: ingest financial and business documents, persist structured rows (transactions) in a database, build a semantic index for question answering, and expose both through a FastAPI backend and a Next.js UI. The “brain” for generation is Google Gemini; retrieval uses local FAISS + sentence-transformers; multi-step reasoning is orchestrated with LangGraph.


Table of contents


What DocSage does

  1. Upload documents (PDF, images, etc.) via the API or UI.
  2. Parse them through an IDP-style pipeline: text extraction (PDF and/or OCR), heuristic classification, and structured field extraction.
  3. StoreDocument rows plus derived Transaction rows in PostgreSQL or SQLite.
  4. Index in FAISS for RAG: per document, an extraction_summary chunk (from structured extracted_data) when present, plus chunked raw_text—each chunk carries chunk_type, chunk_index, and document metadata for grounding in API responses.
  5. Answer questions via POST /api/v1/chat/insights: a LangGraph workflow can route between fast analytic paths, SQL over documents + transactions (with JOIN on document_id), and full retrieve → rerank → grade → synthesize agentic RAG with HyDE-style query rewriting. Optional history on the request enables multi-turn chat (last 20 turns used server-side); synthesis prompts insist on filename / document_id citations when evidence exists.
flowchart LR
subgraph client [Client]
Browser[Browser_Next.js]
end
subgraph api [FastAPI_api]
Routes[routers]
Agent[LangGraph_agent]
RAG[RAGService_FAISS]
DB[(SQLAlchemy_DB)]
LLM[Gemini_API]
end
Browser -->|HTTP_JSON| Routes
Routes --> Agent
Agent --> RAG
Agent --> DB
Agent --> LLM
Routes --> DB
Loading

Repository map

PathRole
api/Python package app: FastAPI entry api/app/main.py, routers, services, agents, models, api/requirements.txt, api/Dockerfile.
api/app/routers/HTTP route modules (analytics, documents, chat, anomalies, compare, receipts, export, insights report).
api/app/services/Business logic: RAG, LLM wrapper, IDP pipeline, SQL tools, extraction_summary helper (extraction_summary.py), insights, anomalies, export, etc.
api/app/agents/LangGraph graph compiler, nodes, state, langgraph_runner.py.
api/app/vectorstore/faiss_store.py: FAISS + embeddings.
api/scripts/Operational scripts (seed, ingest, embeddings, diagnostics)—not started automatically.
web/Next.js 14 App Router UI; entry layout web/src/app/layout.tsx, pages under web/src/app/.
web/src/lib/api.tsTyped fetch helpers against /api/v1.
scripts/run-dev.shLocal dev: optional Postgres, API + web.
docker-compose.postgres.ymlPostgres-only Compose (used by dev script).
docker-compose.ymlPostgres + API image.
docker-compose.prod.ymlProduction-oriented Compose example.
railway.tomlRailway: Dockerfile build, uvicorn start, /health.
.env.exampleTemplate for api/.env (copy into api/).
data/ (under api/ at runtime)Uploads and indexes: raw_docs, embeddings, processed—paths come from config.py (RAW_DOCS_PATH, FAISS_*, etc.) relative to the API working directory. Do not commit large api/data/ trees; keep them local or on a volume.

Quick start (local)

Prerequisites: Node.js, Python 3.11+, optional Docker for Postgres, Google AI Studio API key for LLM features.

From the repository root:

./scripts/run-dev.sh
  • Uses docker-compose.postgres.yml to start Postgres when Docker is available. If Docker is not running, the API and web still start; use USE_SQLITE=true in api/.env for a file DB, or start Docker and rerun.
  • Copies .env.exampleapi/.env once if missing; ensures web/.env.local has NEXT_PUBLIC_API_URL.
  • Recreates api/.venv if broken; installs Python and npm deps; runs uvicorn and npm run dev.

Ports:API_PORT (default 8000), WEB_PORT (default 3000). UI: http://localhost:3000; OpenAPI: http://localhost:8000/docs.

Busy Postgres port:POSTGRES_PORT=5433 ./scripts/run-dev.sh and set POSTGRES_PORT=5433 in api/.env.

Skip Docker:./scripts/run-dev.sh --no-docker

Database initialization

On API startup, api/app/main.py runs a lifespan hook that calls init_db(): SQLAlchemy create_all for models in api/app/models.py. This creates missing tables (e.g. documents, transactions) but does not run Alembic-style migrations for schema changes.


Configuration

API (api/app/config.py + api/.env)

Settings load from environment and optional api/.env. Unknown keys are ignored (extra="ignore") so stale variables do not crash startup.

VariablePurpose
POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB, POSTGRES_HOST, POSTGRES_PORTPostgreSQL connection when USE_SQLITE is false.
USE_SQLITEIf true, DATABASE_URL is SQLite (sqlite:///./docsage.db).
API_HOST, API_PORTUvicorn bind (used when running app.main as __main__).
GOOGLE_API_KEYGemini API key; alias GEMINI_API_KEY.
GOOGLE_AI_MODELPrimary generateContent model id.
GOOGLE_AI_MODEL_FALLBACKSComma-separated fallback model ids (overload / transient errors / unknown primary).
FAISS_INDEX_PATH, FAISS_DOCUMENTS_PATHPaths to FAISS index file and pickle sidecar for chunk metadata.
EMBEDDING_MODELsentence-transformers model name (default all-MiniLM-L6-v2, 384-d vectors).
RAW_DOCS_PATH, PROCESSED_PATHUpload and processed file roots.
CORS_ORIGINSComma-separated browser origins, or * (dev only; avoid in production).
MAX_UPLOAD_MBUpload size cap for document uploads.
DEBUG, LOG_LEVELApp logging / debug flags.

Web

VariablePurpose
NEXT_PUBLIC_API_URLOrigin of the FastAPI server (no trailing slash), e.g. http://127.0.0.1:8000. Used by web/src/lib/api.ts. Must match the host you use in the browser (localhost vs 127.0.0.1) to avoid CORS/preflight issues.
NEXT_PUBLIC_GOOGLE_OAUTH_ENABLEDOptional. Set to true to show “Continue with Google” before the client fetches GET /api/v1/auth/config; otherwise the UI reads that endpoint and only shows the button when the API has OAuth credentials.
NEXT_PUBLIC_SITE_URLOptional canonical site URL for metadata. If unset on Vercel, VERCEL_URL is used in web/src/app/layout.tsx for metadataBase.

Docker Compose: pass GOOGLE_API_KEY, GOOGLE_AI_MODEL, GOOGLE_AI_MODEL_FALLBACKS into the api service (see docker-compose.yml, docker-compose.prod.yml).


Backend architecture

FastAPI application

api/app/main.py constructs the app with:

  • CORS from settings.cors_origins_list.
  • APIRouter subtree mounted at /api/v1 including analytics, anomalies, documents, compare, receipts, export, insights report, chat.
  • LegacyPOST /chat/insights delegating to the same handler as v1 chat.

Persistence

  • SQLAlchemy 2.x engine + SessionLocal in api/app/db.py.
  • Models in api/app/models.py:
    • Document: filename, path, type, raw text, JSON extracted_data, timestamps.
    • Transaction: document_id, date, amount, vendor, category, description, JSON metadata (ORM attribute meta_data to avoid reserved name issues), confidence / correction flags.
    • DocumentCorrection: audit of user corrections.

Routers use Depends(get_db) for request-scoped sessions.

Validation

api/app/schemas.py defines Pydantic models for HTTP I/O. Notable:

  • QueryRequest: query, use_rag, use_sql, optional history (List[ChatMessage] with role and content). The chat router keeps the last 20 turns with non-empty content. use_rag / use_sql bias LangGraph routing (e.g. SQL-only path when RAG is off and keywords suggest aggregation).

Concept glossary

Each item: what the concept is, then how DocSage applies it (files).

Intelligent Document Processing (IDP)

IDP is the class of systems that turn messy documents (PDFs, scans) into structured, machine-usable data—classification, key-value extraction, validation—not just raw text.

DocSage:api/app/services/idp_pipeline.py implements extraction and heuristic classification; the upload path in api/app/routers/documents.py calls parse_document then persists rows. This is “IDP-inspired”: rules + LLM/heuristics rather than a full enterprise IDP product.

OCR (Optical Character Recognition)

OCR recovers text from pixels (photos, scanned pages).

DocSage:pytesseract with Pillow-compatible inputs in extract_text_with_ocr (idp_pipeline.py). The Docker image installs tesseract-ocr (api/Dockerfile) so containers can OCR without extra host setup.

PDF text extraction

Digital PDFs often expose a text layer; extraction without OCR is faster and more accurate.

DocSage:pdfplumber in extract_text_from_pdf walks pages and concatenates extract_text() output (idp_pipeline.py). Image-only PDFs may still need rasterization + OCR (pipeline-dependent).

Heuristic classification and regex extraction

Heuristics use keywords and patterns to guess document type and pull amounts, dates, and vendors without a dedicated ML model per field.

DocSage:classify_document, extract_amounts, and related helpers in idp_pipeline.py; extracted JSON is stored on Document.extracted_data and Transaction rows are synthesized via extract_transactions_from_document (api/scripts/ingest_docs.py) used from the documents router.

Embeddings

An embedding is a dense vector representing text (or other modalities) in a space where semantic similarityvector proximity.

DocSage:SentenceTransformer in api/app/vectorstore/faiss_store.py encodes strings; default model all-MiniLM-L6-v2 produces 384-dimensional vectors (EMBEDDING_MODEL in config). api/scripts/build_embeddings.py walks all Document rows and, for each, emits (1) a single extraction_summary string via extraction_to_index_text (extraction_summary.py) when extracted_data is present, and (2) sliding-window chunks over raw_text when present. Metadata on each vector row includes chunk_type (extraction_summary vs raw_text), chunk_index, total_chunks (for raw splits), and document id/filename—used downstream for rerank sources and UI citations.

Vector store and approximate search (FAISS)

A vector store indexes vectors for nearest-neighbor search (which chunks are closest to the query embedding).

DocSage:FAISSIndexFlatL2—exact L2 search over all vectors (simple, no training). Index and parallel pickle list of metadata are saved to FAISS_INDEX_PATH / FAISS_DOCUMENTS_PATH (relative to API cwd, typically api/data/embeddings/ in local dev). RAGService loads the index if files exist and exposes search(query, k).

RAG (Retrieval-Augmented Generation)

RAG grounds LLM answers in retrieved passages from a corpus instead of parametric memory alone, reducing hallucination on factual questions about your documents.

DocSage:RAGService wraps the FAISS store; LangGraph nodes call search with a rewritten query after HyDE and optional reranking (api/app/agents/langgraph/nodes.py). node_rerank / node_synthesize attach sources: document_id, filename, chunk_index, chunk_type, score for citation-style UX; final synthesis instructs the model to cite filename and document id when evidence exists and to avoid inventing facts when context and SQL are empty.

LangGraph and agentic control flow

LangGraph models an agent as a state machine: nodes (functions) update state; edges (conditional or fixed) choose the next step. “Agentic” here means multiple LLM and tool steps with branching, not a single prompt.

DocSage:build_agent_graph compiles a StateGraph over AgentState (api/app/agents/langgraph/state.py) including optional history for multi-turn prompts. run_agent_pipeline invokes the compiled graph with graph.invoke, then shapes the response for the REST API.

LangChain packages in api/requirements.txt (langchain-core, langchain-community) support the broader ecosystem; application code under app/ imports LangGraph directly rather than high-level LangChain chains.

HyDE (Hypothetical Document Embeddings)

HyDE asks the LLM to draft a fake answer or passage that would answer the question; that text is embedded and used to retrieve real chunks. It often improves recall when the raw user question is short or mismatched to chunk wording.

DocSage:node_hyde_rewrite in nodes.py calls call_llm to produce a hypothetical block; retrieval uses the rewritten text (see also refinement_hint on failed grades). HyDE, grading, SQL generation, and synthesis prompts also receive a compact conversation block from history when the client sends prior turns.

Retrieval depth, cross-encoder reranking, and grading loop

Two-stage retrieval can mean: (1) cheap bi-encoder over many candidates, then (2) cross-encoder scoring query–passage pairs for a top subset. A grader decides if context is good enough or triggers another retrieval loop.

DocSage: constants RETRIEVE_K, RERANK_POOL, RERANK_KEEP, MAX_RETRIEVAL_LOOPS in nodes.py. cross-encoder/ms-marco-MiniLM-L-6-v2 scores pairs. node_grade sets grade_pass; conditional edges in graph.py send failures back to hyde until the cap, then proceed to optional_sqlsynthesize.

SQL grounding (LLM-generated SELECT)

Grounding here means the LLM sees the real table schema and sample rows before emitting read-only SQL executed against your DB.

DocSage:SQLTools introspects transactions for low-level helpers and exposes get_multitable_sql_llm_context()—combined documents + transactions schemas, truncated raw_text previews, and shrunk extracted_data in document samples (SQLite vs Postgres aware). _generate_sql in nodes.py includes that context, the user question, and a short conversation block from history, and explains that transactions.document_id references documents.id (JOIN allowed). node_sql_only and node_optional_sql merge SQL results into answers; optional SQL is keyword-gated (including document/invoice-style terms).

Fast analytic path (metrics shortcut)

Some questions match precomputed aggregates faster than full RAG.

DocSage:node_route checks keyword hints for vendor/category breakdowns and routes to node_metrics_fast, which calls InsightsService (api/app/services/insights.py) and returns without vector search (nodes.py).

Anomaly detection

Anomaly detection flags unusual rows (duplicates, outliers, date oddities).

DocSage:api/app/services/anomaly_detection.py; exposed via api/app/routers/anomalies.py.

Google Gemini (Generative Language API)

Gemini is accessed through RESTgenerateContent (v1beta), not a proprietary SDK requirement in this repo.

DocSage:api/app/services/llm_service.py builds the request with optional systemInstruction, walks a model chain (primary + GOOGLE_AI_MODEL_FALLBACKS), retries 429 / 5xx with backoff and optional Retry-After, skips to the next model on 400 / 404 (e.g. deprecated model id), and returns assistant text or structured error strings.


LangGraph chat pipeline

Graph topology (mermaid)

Mirrors api/app/agents/langgraph/graph.py.

flowchart TD
entry[route_entry]
route[node_route]
metrics[node_metrics_fast]
sqlOnly[node_sql_only]
hyde[node_hyde_rewrite]
retrieve[node_retrieve]
rerank[node_rerank]
grade[node_grade]
sqlOpt[node_optional_sql]
synth[node_synthesize]
endNode[END]
entry --> route
route -->|metrics_fast| metrics
route -->|sql_only| sqlOnly
route -->|agentic_rag| hyde
hyde --> retrieve
retrieve --> rerank
rerank --> grade
grade -->|retry_HyDE_loop| hyde
grade -->|pass_or_cap| sqlOpt
sqlOpt --> synth
metrics --> endNode
sqlOnly --> endNode
synth --> endNode
Loading

The grade → hyde edge is conditional: only when grade_pass is false and retrieval_iteration is below MAX_RETRIEVAL_LOOPS (_grade_next).

HTTP entry and response shape

api/app/routers/chat.py:

  • Lazily constructs a singleton RAGService (loads FAISS if index files exist).
  • run_chat maps request.history to the graph (last 20 non-empty turns).
  • run_chat calls run_agent_pipeline(query, rag, use_rag=..., use_sql=..., history=...).
  • Returns QueryResponse: answer, sources (list of dicts with document_id, filename, chunk_index, chunk_type, score when RAG ran), sql_query, steps, tool_calls (derived from steps for UI convenience).

When RAG is “skipped” in spirit:use_rag=False with SQL-biased routing yields sql_only. use_rag=False also clears sources in the runner output. use_sql=False disables the optional SQL augmentation node path in the graph state.

Other LLM call sites (outside the graph)

Several features call call_llm directly without LangGraph: e.g. insights report generation (api/app/services/insights_generator.py), categorization (api/app/services/categorization.py), HyDE / synthesize / SQL prompt nodes. The graph is the orchestrator for interactive chat; batch/report flows may be linear.


Document ingestion and data flow

  1. POST/api/v1/documents with multipart file (documents.py).

  2. File bytes saved under RAW_DOCS_PATH.

  3. parse_document(path) runs the IDP pipeline (idp_pipeline.py).

  4. Document inserted; extract_transactions_from_document yields dicts → Transaction rows committed.

  5. FAISS is not automatically rebuilt on every upload. After new imports, extraction changes, or IDP tweaks, refresh vectors so extraction_summary and raw_text chunks stay in sync—for example:

    cd api && ./.venv/bin/python scripts/build_embeddings.py

    You can also use RAGService.build_index / add_documents (rag.py) in custom ops; add_documents currently rebuilds the full index for simplicity.


HTTP API surface

All v1 routes are prefixed with /api/v1 unless noted.

TagMethodPathPurpose
analyticsGET/analytics/summaryDashboard KPIs: counts, spend, averages.
GET/analytics/time-seriesTime-bucketed series for charts.
GET/analytics/vendor-statsTop vendors by spend.
GET/analytics/category-breakdownSpend by category.
GET/analytics/spending-forecastSimple forward-looking projection.
GET/analytics/monthly-spendSpend for a given year/month.
anomaliesGET/anomaliesRule-based anomaly list.
chatPOST/chat/insightsAgentic RAG + SQL pipeline. Body: query (required), use_rag, use_sql, optional history (array of { role, content }, server uses last 20 non-empty turns).
compareGET/documents/{document_id}/similarSimilar documents (e.g. shared vendor / join logic in service).
POST/documents/comparePairwise diff / compare (CompareBody).
documentsGET/documentsList documents (filters, pagination).
POST/documentsUpload + parse + persist transactions.
GET/documents/{id}Metadata.
GET/documents/{id}/detailRich detail payload.
GET/documents/{id}/confidenceExtraction confidence signals.
GET/documents/{id}/previewPreview / annotated stream where implemented.
PATCH/documents/{id}Update extracted JSON (DocumentUpdateBody).
exportsGET/exports/excelDownload Excel export blob.
GET/exports/summaryText/markdown summary for export UX.
insights-reportPOST/insights/generate-reportLLM-generated narrative report.
receipt-matchingGET/receipt-matching/unmatchedQueue of unmatched receipts.
POST/receipt-matching/{receipt_doc_id}/matchLink receipt to candidate transaction.

Legacy (no /api/v1 prefix):POST /chat/insights — same body/response as v1 chat.

Interactive docs: /docs (Swagger UI).


Frontend (web)

Pages (examples):dashboard, chat, documents, insights, anomalies, compare, export, receipt-matching; marketing home composes LandingStory.


Dependencies, scripts, and containers

Python (api/requirements.txt) — grouped by role

GroupExamples
HTTPfastapi, uvicorn, python-multipart, pydantic, pydantic-settings
DBsqlalchemy, psycopg2-binary
Agentslanggraph, langchain-core, langchain-community
Vectors / MLfaiss-cpu, sentence-transformers, numpy, pandas
Documentspdfplumber, pytesseract, Pillow, opencv-python, openpyxl
LLM HTTPrequests

Operational scripts (api/scripts/)

Not invoked by default: seed_db.py, ingest_docs.py, build_embeddings.py, migrate_database.py, add_documents_from_folder.py, diagnose_and_fix_transactions.py, download_huggingface_dataset.py, preload_kaggle_invoices.py, etc. Use them manually for migrations, backfills, demo data, and embedding rebuilds.

Docker

api/DockerfileCOPYs from repo root: docker build -f api/Dockerfile . installs Tesseract system packages, Python deps, copies api/app and api/scripts, creates data/ subtrees, runs uvicorn on port 8000.

Compose files wire Postgres + env; see repository root YAMLs.

Railway

railway.toml: Dockerfile builder, uvicorn app.main:app --host 0.0.0.0 --port $PORT, health check /health, restart policy.


Deployment

  • Frontend: deploy subdirectory web/ (e.g. Vercel). Set NEXT_PUBLIC_API_URL to your API’s public origin. Set NEXT_PUBLIC_SITE_URL or rely on VERCEL_URL for metadata (see layout).
  • Backend: container host (Fly, Railway, Cloud Run, etc.) using api/Dockerfile with root build context. Inject GOOGLE_API_KEY, DB URL, CORS_ORIGINS matching the exact browser origin(s) in production.
  • Persistence: mount a volume (or object storage strategy) for api/data/raw_docs, api/data/embeddings, and the SQLite file if used—ephemeral disks lose indexes and uploads on restart.

Authentication and multi-tenancy

DocSage uses JWT-based authentication with email/password registration and optional Google OAuth.

Backend

  • User model in api/app/models.py: email (unique), hashed password (nullable for OAuth-only), optional oauth_provider/oauth_sub.
  • Auth router at /api/v1/auth/ (api/app/routers/auth.py):
    • POST /register — email + password; returns JWT.
    • POST /login — email + password; returns JWT.
    • GET /me — current user from token.
    • GET /google — redirects to Google consent screen.
    • GET /google/callback — exchanges code, upserts user, redirects to frontend with token in URL hash.
  • Dependencyget_current_user in api/app/deps.py protects all non-auth routes.
  • Tenant isolation: every query in documents, transactions, analytics, anomalies, compare, receipts, export, reports, and chat is filtered by user_id.
  • Per-user upload paths: files are stored under data/raw_docs/user_{id}/.
  • Per-user RAG: FAISS indexes live at data/embeddings/user_{id}/.
  • Chat sessions API at /api/v1/chat/sessions (CRUD scoped to current user).

Frontend

  • AuthProvider in web/src/contexts/auth.tsx: stores JWT in localStorage, exposes login, register, logout, setTokenFromOAuth.
  • Bearer token added to all API requests via web/src/lib/api.ts.
  • Route protection in ShellLayout: unauthenticated users are redirected to /login; public routes: /, /login, /register, /auth/callback.
  • Login/Register pages: Google sign-in is shown only when the API reports OAuth is enabled (GET /auth/config) or NEXT_PUBLIC_GOOGLE_OAUTH_ENABLED=true.
  • Chat sessions sync to the server API when authenticated; fall back to localStorage when offline.

Configuration

VariablePurpose
JWT_SECRETSecret for HS256 token signing (change in production).
JWT_EXPIRE_MINUTESToken validity (default 7 days).
GOOGLE_OAUTH_CLIENT_IDGoogle Cloud console client ID (optional).
GOOGLE_OAUTH_CLIENT_SECRETMatching secret.
GOOGLE_OAUTH_REDIRECT_URIMust match console; default http://localhost:8000/api/v1/auth/google/callback.
FRONTEND_URLWhere the OAuth callback redirects with the token hash fragment.

Public GET /api/v1/auth/config (no auth): returns { "google_oauth_enabled": boolean } so the web UI can hide “Continue with Google” when OAuth is not configured on the server.

Migration

Run python scripts/migrate_database.py from api/ to add user_id columns to existing tables and create users / chat_sessions tables. Existing rows without a user_id are hidden from authenticated queries until backfilled.


Limitations and extension points

AreaLimitationPossible extension
FAISSIndexFlatL2 is linear; slow at very large NIVF / HNSW, or managed vector DB (Pinecone, pgvector, …).
Index updatesadd_documents rebuilds whole indexIncremental add, background jobs, versioning.
Schemacreate_all only; no Alembic in treeMigrations for production schema evolution.
Chathistory trimmed to 20 turns with content (chat.py)Configurable cap, thread storage, or rolling summary.
OCRHost must have Tesseract unless using Docker imageCloud OCR APIs, better layout models.
Compare routesMounted at /documents/... alongside document CRUDEnsure route ordering in OpenAPI matches FastAPI resolution for edge IDs.
SecretsNever commit real GOOGLE_API_KEY; rotate if leakedSecret manager, .env gitignored (already).

This README is the authoritative high-level map of the codebase; for line-level behavior, follow the links into api/app/ and web/src/.

About

AI-powered Intelligent Document Processing (IDP) system with RAG, anomaly detection, and natural language insights. Local, zero-cost alternative to AWS Textract + Bedrock.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Repository files navigation

DocSage

DocSage is an agentic document-intelligence system: ingest financial and business documents, persist structured rows (transactions) in a database, build a semantic index for question answering, and expose both through a FastAPI backend and a Next.js UI. The “brain” for generation is Google Gemini; retrieval uses local FAISS + sentence-transformers; multi-step reasoning is orchestrated with LangGraph.


Table of contents


What DocSage does

  1. Upload documents (PDF, images, etc.) via the API or UI.
  2. Parse them through an IDP-style pipeline: text extraction (PDF and/or OCR), heuristic classification, and structured field extraction.
  3. StoreDocument rows plus derived Transaction rows in PostgreSQL or SQLite.
  4. Index in FAISS for RAG: per document, an extraction_summary chunk (from structured extracted_data) when present, plus chunked raw_text—each chunk carries chunk_type, chunk_index, and document metadata for grounding in API responses.
  5. Answer questions via POST /api/v1/chat/insights: a LangGraph workflow can route between fast analytic paths, SQL over documents + transactions (with JOIN on document_id), and full retrieve → rerank → grade → synthesize agentic RAG with HyDE-style query rewriting. Optional history on the request enables multi-turn chat (last 20 turns used server-side); synthesis prompts insist on filename / document_id citations when evidence exists.
flowchart LR
subgraph client [Client]
Browser[Browser_Next.js]
end
subgraph api [FastAPI_api]
Routes[routers]
Agent[LangGraph_agent]
RAG[RAGService_FAISS]
DB[(SQLAlchemy_DB)]
LLM[Gemini_API]
end
Browser -->|HTTP_JSON| Routes
Routes --> Agent
Agent --> RAG
Agent --> DB
Agent --> LLM
Routes --> DB
Loading

Repository map

PathRole
api/Python package app: FastAPI entry api/app/main.py, routers, services, agents, models, api/requirements.txt, api/Dockerfile.
api/app/routers/HTTP route modules (analytics, documents, chat, anomalies, compare, receipts, export, insights report).
api/app/services/Business logic: RAG, LLM wrapper, IDP pipeline, SQL tools, extraction_summary helper (extraction_summary.py), insights, anomalies, export, etc.
api/app/agents/LangGraph graph compiler, nodes, state, langgraph_runner.py.
api/app/vectorstore/faiss_store.py: FAISS + embeddings.
api/scripts/Operational scripts (seed, ingest, embeddings, diagnostics)—not started automatically.
web/Next.js 14 App Router UI; entry layout web/src/app/layout.tsx, pages under web/src/app/.
web/src/lib/api.tsTyped fetch helpers against /api/v1.
scripts/run-dev.shLocal dev: optional Postgres, API + web.
docker-compose.postgres.ymlPostgres-only Compose (used by dev script).
docker-compose.ymlPostgres + API image.
docker-compose.prod.ymlProduction-oriented Compose example.
railway.tomlRailway: Dockerfile build, uvicorn start, /health.
.env.exampleTemplate for api/.env (copy into api/).
data/ (under api/ at runtime)Uploads and indexes: raw_docs, embeddings, processed—paths come from config.py (RAW_DOCS_PATH, FAISS_*, etc.) relative to the API working directory. Do not commit large api/data/ trees; keep them local or on a volume.

Quick start (local)

Prerequisites: Node.js, Python 3.11+, optional Docker for Postgres, Google AI Studio API key for LLM features.

From the repository root:

./scripts/run-dev.sh
  • Uses docker-compose.postgres.yml to start Postgres when Docker is available. If Docker is not running, the API and web still start; use USE_SQLITE=true in api/.env for a file DB, or start Docker and rerun.
  • Copies .env.exampleapi/.env once if missing; ensures web/.env.local has NEXT_PUBLIC_API_URL.
  • Recreates api/.venv if broken; installs Python and npm deps; runs uvicorn and npm run dev.

Ports:API_PORT (default 8000), WEB_PORT (default 3000). UI: http://localhost:3000; OpenAPI: http://localhost:8000/docs.

Busy Postgres port:POSTGRES_PORT=5433 ./scripts/run-dev.sh and set POSTGRES_PORT=5433 in api/.env.

Skip Docker:./scripts/run-dev.sh --no-docker

Database initialization

On API startup, api/app/main.py runs a lifespan hook that calls init_db(): SQLAlchemy create_all for models in api/app/models.py. This creates missing tables (e.g. documents, transactions) but does not run Alembic-style migrations for schema changes.


Configuration

API (api/app/config.py + api/.env)

Settings load from environment and optional api/.env. Unknown keys are ignored (extra="ignore") so stale variables do not crash startup.

VariablePurpose
POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB, POSTGRES_HOST, POSTGRES_PORTPostgreSQL connection when USE_SQLITE is false.
USE_SQLITEIf true, DATABASE_URL is SQLite (sqlite:///./docsage.db).
API_HOST, API_PORTUvicorn bind (used when running app.main as __main__).
GOOGLE_API_KEYGemini API key; alias GEMINI_API_KEY.
GOOGLE_AI_MODELPrimary generateContent model id.
GOOGLE_AI_MODEL_FALLBACKSComma-separated fallback model ids (overload / transient errors / unknown primary).
FAISS_INDEX_PATH, FAISS_DOCUMENTS_PATHPaths to FAISS index file and pickle sidecar for chunk metadata.
EMBEDDING_MODELsentence-transformers model name (default all-MiniLM-L6-v2, 384-d vectors).
RAW_DOCS_PATH, PROCESSED_PATHUpload and processed file roots.
CORS_ORIGINSComma-separated browser origins, or * (dev only; avoid in production).
MAX_UPLOAD_MBUpload size cap for document uploads.
DEBUG, LOG_LEVELApp logging / debug flags.

Web

VariablePurpose
NEXT_PUBLIC_API_URLOrigin of the FastAPI server (no trailing slash), e.g. http://127.0.0.1:8000. Used by web/src/lib/api.ts. Must match the host you use in the browser (localhost vs 127.0.0.1) to avoid CORS/preflight issues.
NEXT_PUBLIC_GOOGLE_OAUTH_ENABLEDOptional. Set to true to show “Continue with Google” before the client fetches GET /api/v1/auth/config; otherwise the UI reads that endpoint and only shows the button when the API has OAuth credentials.
NEXT_PUBLIC_SITE_URLOptional canonical site URL for metadata. If unset on Vercel, VERCEL_URL is used in web/src/app/layout.tsx for metadataBase.

Docker Compose: pass GOOGLE_API_KEY, GOOGLE_AI_MODEL, GOOGLE_AI_MODEL_FALLBACKS into the api service (see docker-compose.yml, docker-compose.prod.yml).


Backend architecture

FastAPI application

api/app/main.py constructs the app with:

  • CORS from settings.cors_origins_list.
  • APIRouter subtree mounted at /api/v1 including analytics, anomalies, documents, compare, receipts, export, insights report, chat.
  • LegacyPOST /chat/insights delegating to the same handler as v1 chat.

Persistence

  • SQLAlchemy 2.x engine + SessionLocal in api/app/db.py.
  • Models in api/app/models.py:
    • Document: filename, path, type, raw text, JSON extracted_data, timestamps.
    • Transaction: document_id, date, amount, vendor, category, description, JSON metadata (ORM attribute meta_data to avoid reserved name issues), confidence / correction flags.
    • DocumentCorrection: audit of user corrections.

Routers use Depends(get_db) for request-scoped sessions.

Validation

api/app/schemas.py defines Pydantic models for HTTP I/O. Notable:

  • QueryRequest: query, use_rag, use_sql, optional history (List[ChatMessage] with role and content). The chat router keeps the last 20 turns with non-empty content. use_rag / use_sql bias LangGraph routing (e.g. SQL-only path when RAG is off and keywords suggest aggregation).

Concept glossary

Each item: what the concept is, then how DocSage applies it (files).

Intelligent Document Processing (IDP)

IDP is the class of systems that turn messy documents (PDFs, scans) into structured, machine-usable data—classification, key-value extraction, validation—not just raw text.

DocSage:api/app/services/idp_pipeline.py implements extraction and heuristic classification; the upload path in api/app/routers/documents.py calls parse_document then persists rows. This is “IDP-inspired”: rules + LLM/heuristics rather than a full enterprise IDP product.

OCR (Optical Character Recognition)

OCR recovers text from pixels (photos, scanned pages).

DocSage:pytesseract with Pillow-compatible inputs in extract_text_with_ocr (idp_pipeline.py). The Docker image installs tesseract-ocr (api/Dockerfile) so containers can OCR without extra host setup.

PDF text extraction

Digital PDFs often expose a text layer; extraction without OCR is faster and more accurate.

DocSage:pdfplumber in extract_text_from_pdf walks pages and concatenates extract_text() output (idp_pipeline.py). Image-only PDFs may still need rasterization + OCR (pipeline-dependent).

Heuristic classification and regex extraction

Heuristics use keywords and patterns to guess document type and pull amounts, dates, and vendors without a dedicated ML model per field.

DocSage:classify_document, extract_amounts, and related helpers in idp_pipeline.py; extracted JSON is stored on Document.extracted_data and Transaction rows are synthesized via extract_transactions_from_document (api/scripts/ingest_docs.py) used from the documents router.

Embeddings

An embedding is a dense vector representing text (or other modalities) in a space where semantic similarityvector proximity.

DocSage:SentenceTransformer in api/app/vectorstore/faiss_store.py encodes strings; default model all-MiniLM-L6-v2 produces 384-dimensional vectors (EMBEDDING_MODEL in config). api/scripts/build_embeddings.py walks all Document rows and, for each, emits (1) a single extraction_summary string via extraction_to_index_text (extraction_summary.py) when extracted_data is present, and (2) sliding-window chunks over raw_text when present. Metadata on each vector row includes chunk_type (extraction_summary vs raw_text), chunk_index, total_chunks (for raw splits), and document id/filename—used downstream for rerank sources and UI citations.

Vector store and approximate search (FAISS)

A vector store indexes vectors for nearest-neighbor search (which chunks are closest to the query embedding).

DocSage:FAISSIndexFlatL2—exact L2 search over all vectors (simple, no training). Index and parallel pickle list of metadata are saved to FAISS_INDEX_PATH / FAISS_DOCUMENTS_PATH (relative to API cwd, typically api/data/embeddings/ in local dev). RAGService loads the index if files exist and exposes search(query, k).

RAG (Retrieval-Augmented Generation)

RAG grounds LLM answers in retrieved passages from a corpus instead of parametric memory alone, reducing hallucination on factual questions about your documents.

DocSage:RAGService wraps the FAISS store; LangGraph nodes call search with a rewritten query after HyDE and optional reranking (api/app/agents/langgraph/nodes.py). node_rerank / node_synthesize attach sources: document_id, filename, chunk_index, chunk_type, score for citation-style UX; final synthesis instructs the model to cite filename and document id when evidence exists and to avoid inventing facts when context and SQL are empty.

LangGraph and agentic control flow

LangGraph models an agent as a state machine: nodes (functions) update state; edges (conditional or fixed) choose the next step. “Agentic” here means multiple LLM and tool steps with branching, not a single prompt.

DocSage:build_agent_graph compiles a StateGraph over AgentState (api/app/agents/langgraph/state.py) including optional history for multi-turn prompts. run_agent_pipeline invokes the compiled graph with graph.invoke, then shapes the response for the REST API.

LangChain packages in api/requirements.txt (langchain-core, langchain-community) support the broader ecosystem; application code under app/ imports LangGraph directly rather than high-level LangChain chains.

HyDE (Hypothetical Document Embeddings)

HyDE asks the LLM to draft a fake answer or passage that would answer the question; that text is embedded and used to retrieve real chunks. It often improves recall when the raw user question is short or mismatched to chunk wording.

DocSage:node_hyde_rewrite in nodes.py calls call_llm to produce a hypothetical block; retrieval uses the rewritten text (see also refinement_hint on failed grades). HyDE, grading, SQL generation, and synthesis prompts also receive a compact conversation block from history when the client sends prior turns.

Retrieval depth, cross-encoder reranking, and grading loop

Two-stage retrieval can mean: (1) cheap bi-encoder over many candidates, then (2) cross-encoder scoring query–passage pairs for a top subset. A grader decides if context is good enough or triggers another retrieval loop.

DocSage: constants RETRIEVE_K, RERANK_POOL, RERANK_KEEP, MAX_RETRIEVAL_LOOPS in nodes.py. cross-encoder/ms-marco-MiniLM-L-6-v2 scores pairs. node_grade sets grade_pass; conditional edges in graph.py send failures back to hyde until the cap, then proceed to optional_sqlsynthesize.

SQL grounding (LLM-generated SELECT)

Grounding here means the LLM sees the real table schema and sample rows before emitting read-only SQL executed against your DB.

DocSage:SQLTools introspects transactions for low-level helpers and exposes get_multitable_sql_llm_context()—combined documents + transactions schemas, truncated raw_text previews, and shrunk extracted_data in document samples (SQLite vs Postgres aware). _generate_sql in nodes.py includes that context, the user question, and a short conversation block from history, and explains that transactions.document_id references documents.id (JOIN allowed). node_sql_only and node_optional_sql merge SQL results into answers; optional SQL is keyword-gated (including document/invoice-style terms).

Fast analytic path (metrics shortcut)

Some questions match precomputed aggregates faster than full RAG.

DocSage:node_route checks keyword hints for vendor/category breakdowns and routes to node_metrics_fast, which calls InsightsService (api/app/services/insights.py) and returns without vector search (nodes.py).

Anomaly detection

Anomaly detection flags unusual rows (duplicates, outliers, date oddities).

DocSage:api/app/services/anomaly_detection.py; exposed via api/app/routers/anomalies.py.

Google Gemini (Generative Language API)

Gemini is accessed through RESTgenerateContent (v1beta), not a proprietary SDK requirement in this repo.

DocSage:api/app/services/llm_service.py builds the request with optional systemInstruction, walks a model chain (primary + GOOGLE_AI_MODEL_FALLBACKS), retries 429 / 5xx with backoff and optional Retry-After, skips to the next model on 400 / 404 (e.g. deprecated model id), and returns assistant text or structured error strings.


LangGraph chat pipeline

Graph topology (mermaid)

Mirrors api/app/agents/langgraph/graph.py.

flowchart TD
entry[route_entry]
route[node_route]
metrics[node_metrics_fast]
sqlOnly[node_sql_only]
hyde[node_hyde_rewrite]
retrieve[node_retrieve]
rerank[node_rerank]
grade[node_grade]
sqlOpt[node_optional_sql]
synth[node_synthesize]
endNode[END]
entry --> route
route -->|metrics_fast| metrics
route -->|sql_only| sqlOnly
route -->|agentic_rag| hyde
hyde --> retrieve
retrieve --> rerank
rerank --> grade
grade -->|retry_HyDE_loop| hyde
grade -->|pass_or_cap| sqlOpt
sqlOpt --> synth
metrics --> endNode
sqlOnly --> endNode
synth --> endNode
Loading

The grade → hyde edge is conditional: only when grade_pass is false and retrieval_iteration is below MAX_RETRIEVAL_LOOPS (_grade_next).

HTTP entry and response shape

api/app/routers/chat.py:

  • Lazily constructs a singleton RAGService (loads FAISS if index files exist).
  • run_chat maps request.history to the graph (last 20 non-empty turns).
  • run_chat calls run_agent_pipeline(query, rag, use_rag=..., use_sql=..., history=...).
  • Returns QueryResponse: answer, sources (list of dicts with document_id, filename, chunk_index, chunk_type, score when RAG ran), sql_query, steps, tool_calls (derived from steps for UI convenience).

When RAG is “skipped” in spirit:use_rag=False with SQL-biased routing yields sql_only. use_rag=False also clears sources in the runner output. use_sql=False disables the optional SQL augmentation node path in the graph state.

Other LLM call sites (outside the graph)

Several features call call_llm directly without LangGraph: e.g. insights report generation (api/app/services/insights_generator.py), categorization (api/app/services/categorization.py), HyDE / synthesize / SQL prompt nodes. The graph is the orchestrator for interactive chat; batch/report flows may be linear.


Document ingestion and data flow

  1. POST/api/v1/documents with multipart file (documents.py).

  2. File bytes saved under RAW_DOCS_PATH.

  3. parse_document(path) runs the IDP pipeline (idp_pipeline.py).

  4. Document inserted; extract_transactions_from_document yields dicts → Transaction rows committed.

  5. FAISS is not automatically rebuilt on every upload. After new imports, extraction changes, or IDP tweaks, refresh vectors so extraction_summary and raw_text chunks stay in sync—for example:

    cd api && ./.venv/bin/python scripts/build_embeddings.py

    You can also use RAGService.build_index / add_documents (rag.py) in custom ops; add_documents currently rebuilds the full index for simplicity.


HTTP API surface

All v1 routes are prefixed with /api/v1 unless noted.

TagMethodPathPurpose
analyticsGET/analytics/summaryDashboard KPIs: counts, spend, averages.
GET/analytics/time-seriesTime-bucketed series for charts.
GET/analytics/vendor-statsTop vendors by spend.
GET/analytics/category-breakdownSpend by category.
GET/analytics/spending-forecastSimple forward-looking projection.
GET/analytics/monthly-spendSpend for a given year/month.
anomaliesGET/anomaliesRule-based anomaly list.
chatPOST/chat/insightsAgentic RAG + SQL pipeline. Body: query (required), use_rag, use_sql, optional history (array of { role, content }, server uses last 20 non-empty turns).
compareGET/documents/{document_id}/similarSimilar documents (e.g. shared vendor / join logic in service).
POST/documents/comparePairwise diff / compare (CompareBody).
documentsGET/documentsList documents (filters, pagination).
POST/documentsUpload + parse + persist transactions.
GET/documents/{id}Metadata.
GET/documents/{id}/detailRich detail payload.
GET/documents/{id}/confidenceExtraction confidence signals.
GET/documents/{id}/previewPreview / annotated stream where implemented.
PATCH/documents/{id}Update extracted JSON (DocumentUpdateBody).
exportsGET/exports/excelDownload Excel export blob.
GET/exports/summaryText/markdown summary for export UX.
insights-reportPOST/insights/generate-reportLLM-generated narrative report.
receipt-matchingGET/receipt-matching/unmatchedQueue of unmatched receipts.
POST/receipt-matching/{receipt_doc_id}/matchLink receipt to candidate transaction.

Legacy (no /api/v1 prefix):POST /chat/insights — same body/response as v1 chat.

Interactive docs: /docs (Swagger UI).


Frontend (web)

Pages (examples):dashboard, chat, documents, insights, anomalies, compare, export, receipt-matching; marketing home composes LandingStory.


Dependencies, scripts, and containers

Python (api/requirements.txt) — grouped by role

GroupExamples
HTTPfastapi, uvicorn, python-multipart, pydantic, pydantic-settings
DBsqlalchemy, psycopg2-binary
Agentslanggraph, langchain-core, langchain-community
Vectors / MLfaiss-cpu, sentence-transformers, numpy, pandas
Documentspdfplumber, pytesseract, Pillow, opencv-python, openpyxl
LLM HTTPrequests

Operational scripts (api/scripts/)

Not invoked by default: seed_db.py, ingest_docs.py, build_embeddings.py, migrate_database.py, add_documents_from_folder.py, diagnose_and_fix_transactions.py, download_huggingface_dataset.py, preload_kaggle_invoices.py, etc. Use them manually for migrations, backfills, demo data, and embedding rebuilds.

Docker

api/DockerfileCOPYs from repo root: docker build -f api/Dockerfile . installs Tesseract system packages, Python deps, copies api/app and api/scripts, creates data/ subtrees, runs uvicorn on port 8000.

Compose files wire Postgres + env; see repository root YAMLs.

Railway

railway.toml: Dockerfile builder, uvicorn app.main:app --host 0.0.0.0 --port $PORT, health check /health, restart policy.


Deployment

  • Frontend: deploy subdirectory web/ (e.g. Vercel). Set NEXT_PUBLIC_API_URL to your API’s public origin. Set NEXT_PUBLIC_SITE_URL or rely on VERCEL_URL for metadata (see layout).
  • Backend: container host (Fly, Railway, Cloud Run, etc.) using api/Dockerfile with root build context. Inject GOOGLE_API_KEY, DB URL, CORS_ORIGINS matching the exact browser origin(s) in production.
  • Persistence: mount a volume (or object storage strategy) for api/data/raw_docs, api/data/embeddings, and the SQLite file if used—ephemeral disks lose indexes and uploads on restart.

Authentication and multi-tenancy

DocSage uses JWT-based authentication with email/password registration and optional Google OAuth.

Backend

  • User model in api/app/models.py: email (unique), hashed password (nullable for OAuth-only), optional oauth_provider/oauth_sub.
  • Auth router at /api/v1/auth/ (api/app/routers/auth.py):
    • POST /register — email + password; returns JWT.
    • POST /login — email + password; returns JWT.
    • GET /me — current user from token.
    • GET /google — redirects to Google consent screen.
    • GET /google/callback — exchanges code, upserts user, redirects to frontend with token in URL hash.
  • Dependencyget_current_user in api/app/deps.py protects all non-auth routes.
  • Tenant isolation: every query in documents, transactions, analytics, anomalies, compare, receipts, export, reports, and chat is filtered by user_id.
  • Per-user upload paths: files are stored under data/raw_docs/user_{id}/.
  • Per-user RAG: FAISS indexes live at data/embeddings/user_{id}/.
  • Chat sessions API at /api/v1/chat/sessions (CRUD scoped to current user).

Frontend

  • AuthProvider in web/src/contexts/auth.tsx: stores JWT in localStorage, exposes login, register, logout, setTokenFromOAuth.
  • Bearer token added to all API requests via web/src/lib/api.ts.
  • Route protection in ShellLayout: unauthenticated users are redirected to /login; public routes: /, /login, /register, /auth/callback.
  • Login/Register pages: Google sign-in is shown only when the API reports OAuth is enabled (GET /auth/config) or NEXT_PUBLIC_GOOGLE_OAUTH_ENABLED=true.
  • Chat sessions sync to the server API when authenticated; fall back to localStorage when offline.

Configuration

VariablePurpose
JWT_SECRETSecret for HS256 token signing (change in production).
JWT_EXPIRE_MINUTESToken validity (default 7 days).
GOOGLE_OAUTH_CLIENT_IDGoogle Cloud console client ID (optional).
GOOGLE_OAUTH_CLIENT_SECRETMatching secret.
GOOGLE_OAUTH_REDIRECT_URIMust match console; default http://localhost:8000/api/v1/auth/google/callback.
FRONTEND_URLWhere the OAuth callback redirects with the token hash fragment.

Public GET /api/v1/auth/config (no auth): returns { "google_oauth_enabled": boolean } so the web UI can hide “Continue with Google” when OAuth is not configured on the server.

Migration

Run python scripts/migrate_database.py from api/ to add user_id columns to existing tables and create users / chat_sessions tables. Existing rows without a user_id are hidden from authenticated queries until backfilled.


Limitations and extension points

AreaLimitationPossible extension
FAISSIndexFlatL2 is linear; slow at very large NIVF / HNSW, or managed vector DB (Pinecone, pgvector, …).
Index updatesadd_documents rebuilds whole indexIncremental add, background jobs, versioning.
Schemacreate_all only; no Alembic in treeMigrations for production schema evolution.
Chathistory trimmed to 20 turns with content (chat.py)Configurable cap, thread storage, or rolling summary.
OCRHost must have Tesseract unless using Docker imageCloud OCR APIs, better layout models.
Compare routesMounted at /documents/... alongside document CRUDEnsure route ordering in OpenAPI matches FastAPI resolution for edge IDs.
SecretsNever commit real GOOGLE_API_KEY; rotate if leakedSecret manager, .env gitignored (already).

This README is the authoritative high-level map of the codebase; for line-level behavior, follow the links into api/app/ and web/src/.

About

AI-powered Intelligent Document Processing (IDP) system with RAG, anomaly detection, and natural language insights. Local, zero-cost alternative to AWS Textract + Bedrock.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Repository files navigation

DocSage

DocSage is an agentic document-intelligence system: ingest financial and business documents, persist structured rows (transactions) in a database, build a semantic index for question answering, and expose both through a FastAPI backend and a Next.js UI. The “brain” for generation is Google Gemini; retrieval uses local FAISS + sentence-transformers; multi-step reasoning is orchestrated with LangGraph.


Table of contents


What DocSage does

  1. Upload documents (PDF, images, etc.) via the API or UI.
  2. Parse them through an IDP-style pipeline: text extraction (PDF and/or OCR), heuristic classification, and structured field extraction.
  3. StoreDocument rows plus derived Transaction rows in PostgreSQL or SQLite.
  4. Index in FAISS for RAG: per document, an extraction_summary chunk (from structured extracted_data) when present, plus chunked raw_text—each chunk carries chunk_type, chunk_index, and document metadata for grounding in API responses.
  5. Answer questions via POST /api/v1/chat/insights: a LangGraph workflow can route between fast analytic paths, SQL over documents + transactions (with JOIN on document_id), and full retrieve → rerank → grade → synthesize agentic RAG with HyDE-style query rewriting. Optional history on the request enables multi-turn chat (last 20 turns used server-side); synthesis prompts insist on filename / document_id citations when evidence exists.
flowchart LR
subgraph client [Client]
Browser[Browser_Next.js]
end
subgraph api [FastAPI_api]
Routes[routers]
Agent[LangGraph_agent]
RAG[RAGService_FAISS]
DB[(SQLAlchemy_DB)]
LLM[Gemini_API]
end
Browser -->|HTTP_JSON| Routes
Routes --> Agent
Agent --> RAG
Agent --> DB
Agent --> LLM
Routes --> DB
Loading

Repository map

PathRole
api/Python package app: FastAPI entry api/app/main.py, routers, services, agents, models, api/requirements.txt, api/Dockerfile.
api/app/routers/HTTP route modules (analytics, documents, chat, anomalies, compare, receipts, export, insights report).
api/app/services/Business logic: RAG, LLM wrapper, IDP pipeline, SQL tools, extraction_summary helper (extraction_summary.py), insights, anomalies, export, etc.
api/app/agents/LangGraph graph compiler, nodes, state, langgraph_runner.py.
api/app/vectorstore/faiss_store.py: FAISS + embeddings.
api/scripts/Operational scripts (seed, ingest, embeddings, diagnostics)—not started automatically.
web/Next.js 14 App Router UI; entry layout web/src/app/layout.tsx, pages under web/src/app/.
web/src/lib/api.tsTyped fetch helpers against /api/v1.
scripts/run-dev.shLocal dev: optional Postgres, API + web.
docker-compose.postgres.ymlPostgres-only Compose (used by dev script).
docker-compose.ymlPostgres + API image.
docker-compose.prod.ymlProduction-oriented Compose example.
railway.tomlRailway: Dockerfile build, uvicorn start, /health.
.env.exampleTemplate for api/.env (copy into api/).
data/ (under api/ at runtime)Uploads and indexes: raw_docs, embeddings, processed—paths come from config.py (RAW_DOCS_PATH, FAISS_*, etc.) relative to the API working directory. Do not commit large api/data/ trees; keep them local or on a volume.

Quick start (local)

Prerequisites: Node.js, Python 3.11+, optional Docker for Postgres, Google AI Studio API key for LLM features.

From the repository root:

./scripts/run-dev.sh
  • Uses docker-compose.postgres.yml to start Postgres when Docker is available. If Docker is not running, the API and web still start; use USE_SQLITE=true in api/.env for a file DB, or start Docker and rerun.
  • Copies .env.exampleapi/.env once if missing; ensures web/.env.local has NEXT_PUBLIC_API_URL.
  • Recreates api/.venv if broken; installs Python and npm deps; runs uvicorn and npm run dev.

Ports:API_PORT (default 8000), WEB_PORT (default 3000). UI: http://localhost:3000; OpenAPI: http://localhost:8000/docs.

Busy Postgres port:POSTGRES_PORT=5433 ./scripts/run-dev.sh and set POSTGRES_PORT=5433 in api/.env.

Skip Docker:./scripts/run-dev.sh --no-docker

Database initialization

On API startup, api/app/main.py runs a lifespan hook that calls init_db(): SQLAlchemy create_all for models in api/app/models.py. This creates missing tables (e.g. documents, transactions) but does not run Alembic-style migrations for schema changes.


Configuration

API (api/app/config.py + api/.env)

Settings load from environment and optional api/.env. Unknown keys are ignored (extra="ignore") so stale variables do not crash startup.

VariablePurpose
POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB, POSTGRES_HOST, POSTGRES_PORTPostgreSQL connection when USE_SQLITE is false.
USE_SQLITEIf true, DATABASE_URL is SQLite (sqlite:///./docsage.db).
API_HOST, API_PORTUvicorn bind (used when running app.main as __main__).
GOOGLE_API_KEYGemini API key; alias GEMINI_API_KEY.
GOOGLE_AI_MODELPrimary generateContent model id.
GOOGLE_AI_MODEL_FALLBACKSComma-separated fallback model ids (overload / transient errors / unknown primary).
FAISS_INDEX_PATH, FAISS_DOCUMENTS_PATHPaths to FAISS index file and pickle sidecar for chunk metadata.
EMBEDDING_MODELsentence-transformers model name (default all-MiniLM-L6-v2, 384-d vectors).
RAW_DOCS_PATH, PROCESSED_PATHUpload and processed file roots.
CORS_ORIGINSComma-separated browser origins, or * (dev only; avoid in production).
MAX_UPLOAD_MBUpload size cap for document uploads.
DEBUG, LOG_LEVELApp logging / debug flags.

Web

VariablePurpose
NEXT_PUBLIC_API_URLOrigin of the FastAPI server (no trailing slash), e.g. http://127.0.0.1:8000. Used by web/src/lib/api.ts. Must match the host you use in the browser (localhost vs 127.0.0.1) to avoid CORS/preflight issues.
NEXT_PUBLIC_GOOGLE_OAUTH_ENABLEDOptional. Set to true to show “Continue with Google” before the client fetches GET /api/v1/auth/config; otherwise the UI reads that endpoint and only shows the button when the API has OAuth credentials.
NEXT_PUBLIC_SITE_URLOptional canonical site URL for metadata. If unset on Vercel, VERCEL_URL is used in web/src/app/layout.tsx for metadataBase.

Docker Compose: pass GOOGLE_API_KEY, GOOGLE_AI_MODEL, GOOGLE_AI_MODEL_FALLBACKS into the api service (see docker-compose.yml, docker-compose.prod.yml).


Backend architecture

FastAPI application

api/app/main.py constructs the app with:

  • CORS from settings.cors_origins_list.
  • APIRouter subtree mounted at /api/v1 including analytics, anomalies, documents, compare, receipts, export, insights report, chat.
  • LegacyPOST /chat/insights delegating to the same handler as v1 chat.

Persistence

  • SQLAlchemy 2.x engine + SessionLocal in api/app/db.py.
  • Models in api/app/models.py:
    • Document: filename, path, type, raw text, JSON extracted_data, timestamps.
    • Transaction: document_id, date, amount, vendor, category, description, JSON metadata (ORM attribute meta_data to avoid reserved name issues), confidence / correction flags.
    • DocumentCorrection: audit of user corrections.

Routers use Depends(get_db) for request-scoped sessions.

Validation

api/app/schemas.py defines Pydantic models for HTTP I/O. Notable:

  • QueryRequest: query, use_rag, use_sql, optional history (List[ChatMessage] with role and content). The chat router keeps the last 20 turns with non-empty content. use_rag / use_sql bias LangGraph routing (e.g. SQL-only path when RAG is off and keywords suggest aggregation).

Concept glossary

Each item: what the concept is, then how DocSage applies it (files).

Intelligent Document Processing (IDP)

IDP is the class of systems that turn messy documents (PDFs, scans) into structured, machine-usable data—classification, key-value extraction, validation—not just raw text.

DocSage:api/app/services/idp_pipeline.py implements extraction and heuristic classification; the upload path in api/app/routers/documents.py calls parse_document then persists rows. This is “IDP-inspired”: rules + LLM/heuristics rather than a full enterprise IDP product.

OCR (Optical Character Recognition)

OCR recovers text from pixels (photos, scanned pages).

DocSage:pytesseract with Pillow-compatible inputs in extract_text_with_ocr (idp_pipeline.py). The Docker image installs tesseract-ocr (api/Dockerfile) so containers can OCR without extra host setup.

PDF text extraction

Digital PDFs often expose a text layer; extraction without OCR is faster and more accurate.

DocSage:pdfplumber in extract_text_from_pdf walks pages and concatenates extract_text() output (idp_pipeline.py). Image-only PDFs may still need rasterization + OCR (pipeline-dependent).

Heuristic classification and regex extraction

Heuristics use keywords and patterns to guess document type and pull amounts, dates, and vendors without a dedicated ML model per field.

DocSage:classify_document, extract_amounts, and related helpers in idp_pipeline.py; extracted JSON is stored on Document.extracted_data and Transaction rows are synthesized via extract_transactions_from_document (api/scripts/ingest_docs.py) used from the documents router.

Embeddings

An embedding is a dense vector representing text (or other modalities) in a space where semantic similarityvector proximity.

DocSage:SentenceTransformer in api/app/vectorstore/faiss_store.py encodes strings; default model all-MiniLM-L6-v2 produces 384-dimensional vectors (EMBEDDING_MODEL in config). api/scripts/build_embeddings.py walks all Document rows and, for each, emits (1) a single extraction_summary string via extraction_to_index_text (extraction_summary.py) when extracted_data is present, and (2) sliding-window chunks over raw_text when present. Metadata on each vector row includes chunk_type (extraction_summary vs raw_text), chunk_index, total_chunks (for raw splits), and document id/filename—used downstream for rerank sources and UI citations.

Vector store and approximate search (FAISS)

A vector store indexes vectors for nearest-neighbor search (which chunks are closest to the query embedding).

DocSage:FAISSIndexFlatL2—exact L2 search over all vectors (simple, no training). Index and parallel pickle list of metadata are saved to FAISS_INDEX_PATH / FAISS_DOCUMENTS_PATH (relative to API cwd, typically api/data/embeddings/ in local dev). RAGService loads the index if files exist and exposes search(query, k).

RAG (Retrieval-Augmented Generation)

RAG grounds LLM answers in retrieved passages from a corpus instead of parametric memory alone, reducing hallucination on factual questions about your documents.

DocSage:RAGService wraps the FAISS store; LangGraph nodes call search with a rewritten query after HyDE and optional reranking (api/app/agents/langgraph/nodes.py). node_rerank / node_synthesize attach sources: document_id, filename, chunk_index, chunk_type, score for citation-style UX; final synthesis instructs the model to cite filename and document id when evidence exists and to avoid inventing facts when context and SQL are empty.

LangGraph and agentic control flow

LangGraph models an agent as a state machine: nodes (functions) update state; edges (conditional or fixed) choose the next step. “Agentic” here means multiple LLM and tool steps with branching, not a single prompt.

DocSage:build_agent_graph compiles a StateGraph over AgentState (api/app/agents/langgraph/state.py) including optional history for multi-turn prompts. run_agent_pipeline invokes the compiled graph with graph.invoke, then shapes the response for the REST API.

LangChain packages in api/requirements.txt (langchain-core, langchain-community) support the broader ecosystem; application code under app/ imports LangGraph directly rather than high-level LangChain chains.

HyDE (Hypothetical Document Embeddings)

HyDE asks the LLM to draft a fake answer or passage that would answer the question; that text is embedded and used to retrieve real chunks. It often improves recall when the raw user question is short or mismatched to chunk wording.

DocSage:node_hyde_rewrite in nodes.py calls call_llm to produce a hypothetical block; retrieval uses the rewritten text (see also refinement_hint on failed grades). HyDE, grading, SQL generation, and synthesis prompts also receive a compact conversation block from history when the client sends prior turns.

Retrieval depth, cross-encoder reranking, and grading loop

Two-stage retrieval can mean: (1) cheap bi-encoder over many candidates, then (2) cross-encoder scoring query–passage pairs for a top subset. A grader decides if context is good enough or triggers another retrieval loop.

DocSage: constants RETRIEVE_K, RERANK_POOL, RERANK_KEEP, MAX_RETRIEVAL_LOOPS in nodes.py. cross-encoder/ms-marco-MiniLM-L-6-v2 scores pairs. node_grade sets grade_pass; conditional edges in graph.py send failures back to hyde until the cap, then proceed to optional_sqlsynthesize.

SQL grounding (LLM-generated SELECT)

Grounding here means the LLM sees the real table schema and sample rows before emitting read-only SQL executed against your DB.

DocSage:SQLTools introspects transactions for low-level helpers and exposes get_multitable_sql_llm_context()—combined documents + transactions schemas, truncated raw_text previews, and shrunk extracted_data in document samples (SQLite vs Postgres aware). _generate_sql in nodes.py includes that context, the user question, and a short conversation block from history, and explains that transactions.document_id references documents.id (JOIN allowed). node_sql_only and node_optional_sql merge SQL results into answers; optional SQL is keyword-gated (including document/invoice-style terms).

Fast analytic path (metrics shortcut)

Some questions match precomputed aggregates faster than full RAG.

DocSage:node_route checks keyword hints for vendor/category breakdowns and routes to node_metrics_fast, which calls InsightsService (api/app/services/insights.py) and returns without vector search (nodes.py).

Anomaly detection

Anomaly detection flags unusual rows (duplicates, outliers, date oddities).

DocSage:api/app/services/anomaly_detection.py; exposed via api/app/routers/anomalies.py.

Google Gemini (Generative Language API)

Gemini is accessed through RESTgenerateContent (v1beta), not a proprietary SDK requirement in this repo.

DocSage:api/app/services/llm_service.py builds the request with optional systemInstruction, walks a model chain (primary + GOOGLE_AI_MODEL_FALLBACKS), retries 429 / 5xx with backoff and optional Retry-After, skips to the next model on 400 / 404 (e.g. deprecated model id), and returns assistant text or structured error strings.


LangGraph chat pipeline

Graph topology (mermaid)

Mirrors api/app/agents/langgraph/graph.py.

flowchart TD
entry[route_entry]
route[node_route]
metrics[node_metrics_fast]
sqlOnly[node_sql_only]
hyde[node_hyde_rewrite]
retrieve[node_retrieve]
rerank[node_rerank]
grade[node_grade]
sqlOpt[node_optional_sql]
synth[node_synthesize]
endNode[END]
entry --> route
route -->|metrics_fast| metrics
route -->|sql_only| sqlOnly
route -->|agentic_rag| hyde
hyde --> retrieve
retrieve --> rerank
rerank --> grade
grade -->|retry_HyDE_loop| hyde
grade -->|pass_or_cap| sqlOpt
sqlOpt --> synth
metrics --> endNode
sqlOnly --> endNode
synth --> endNode
Loading

The grade → hyde edge is conditional: only when grade_pass is false and retrieval_iteration is below MAX_RETRIEVAL_LOOPS (_grade_next).

HTTP entry and response shape

api/app/routers/chat.py:

  • Lazily constructs a singleton RAGService (loads FAISS if index files exist).
  • run_chat maps request.history to the graph (last 20 non-empty turns).
  • run_chat calls run_agent_pipeline(query, rag, use_rag=..., use_sql=..., history=...).
  • Returns QueryResponse: answer, sources (list of dicts with document_id, filename, chunk_index, chunk_type, score when RAG ran), sql_query, steps, tool_calls (derived from steps for UI convenience).

When RAG is “skipped” in spirit:use_rag=False with SQL-biased routing yields sql_only. use_rag=False also clears sources in the runner output. use_sql=False disables the optional SQL augmentation node path in the graph state.

Other LLM call sites (outside the graph)

Several features call call_llm directly without LangGraph: e.g. insights report generation (api/app/services/insights_generator.py), categorization (api/app/services/categorization.py), HyDE / synthesize / SQL prompt nodes. The graph is the orchestrator for interactive chat; batch/report flows may be linear.


Document ingestion and data flow

  1. POST/api/v1/documents with multipart file (documents.py).

  2. File bytes saved under RAW_DOCS_PATH.

  3. parse_document(path) runs the IDP pipeline (idp_pipeline.py).

  4. Document inserted; extract_transactions_from_document yields dicts → Transaction rows committed.

  5. FAISS is not automatically rebuilt on every upload. After new imports, extraction changes, or IDP tweaks, refresh vectors so extraction_summary and raw_text chunks stay in sync—for example:

    cd api && ./.venv/bin/python scripts/build_embeddings.py

    You can also use RAGService.build_index / add_documents (rag.py) in custom ops; add_documents currently rebuilds the full index for simplicity.


HTTP API surface

All v1 routes are prefixed with /api/v1 unless noted.

TagMethodPathPurpose
analyticsGET/analytics/summaryDashboard KPIs: counts, spend, averages.
GET/analytics/time-seriesTime-bucketed series for charts.
GET/analytics/vendor-statsTop vendors by spend.
GET/analytics/category-breakdownSpend by category.
GET/analytics/spending-forecastSimple forward-looking projection.
GET/analytics/monthly-spendSpend for a given year/month.
anomaliesGET/anomaliesRule-based anomaly list.
chatPOST/chat/insightsAgentic RAG + SQL pipeline. Body: query (required), use_rag, use_sql, optional history (array of { role, content }, server uses last 20 non-empty turns).
compareGET/documents/{document_id}/similarSimilar documents (e.g. shared vendor / join logic in service).
POST/documents/comparePairwise diff / compare (CompareBody).
documentsGET/documentsList documents (filters, pagination).
POST/documentsUpload + parse + persist transactions.
GET/documents/{id}Metadata.
GET/documents/{id}/detailRich detail payload.
GET/documents/{id}/confidenceExtraction confidence signals.
GET/documents/{id}/previewPreview / annotated stream where implemented.
PATCH/documents/{id}Update extracted JSON (DocumentUpdateBody).
exportsGET/exports/excelDownload Excel export blob.
GET/exports/summaryText/markdown summary for export UX.
insights-reportPOST/insights/generate-reportLLM-generated narrative report.
receipt-matchingGET/receipt-matching/unmatchedQueue of unmatched receipts.
POST/receipt-matching/{receipt_doc_id}/matchLink receipt to candidate transaction.

Legacy (no /api/v1 prefix):POST /chat/insights — same body/response as v1 chat.

Interactive docs: /docs (Swagger UI).


Frontend (web)

Pages (examples):dashboard, chat, documents, insights, anomalies, compare, export, receipt-matching; marketing home composes LandingStory.


Dependencies, scripts, and containers

Python (api/requirements.txt) — grouped by role

GroupExamples
HTTPfastapi, uvicorn, python-multipart, pydantic, pydantic-settings
DBsqlalchemy, psycopg2-binary
Agentslanggraph, langchain-core, langchain-community
Vectors / MLfaiss-cpu, sentence-transformers, numpy, pandas
Documentspdfplumber, pytesseract, Pillow, opencv-python, openpyxl
LLM HTTPrequests

Operational scripts (api/scripts/)

Not invoked by default: seed_db.py, ingest_docs.py, build_embeddings.py, migrate_database.py, add_documents_from_folder.py, diagnose_and_fix_transactions.py, download_huggingface_dataset.py, preload_kaggle_invoices.py, etc. Use them manually for migrations, backfills, demo data, and embedding rebuilds.

Docker

api/DockerfileCOPYs from repo root: docker build -f api/Dockerfile . installs Tesseract system packages, Python deps, copies api/app and api/scripts, creates data/ subtrees, runs uvicorn on port 8000.

Compose files wire Postgres + env; see repository root YAMLs.

Railway

railway.toml: Dockerfile builder, uvicorn app.main:app --host 0.0.0.0 --port $PORT, health check /health, restart policy.


Deployment

  • Frontend: deploy subdirectory web/ (e.g. Vercel). Set NEXT_PUBLIC_API_URL to your API’s public origin. Set NEXT_PUBLIC_SITE_URL or rely on VERCEL_URL for metadata (see layout).
  • Backend: container host (Fly, Railway, Cloud Run, etc.) using api/Dockerfile with root build context. Inject GOOGLE_API_KEY, DB URL, CORS_ORIGINS matching the exact browser origin(s) in production.
  • Persistence: mount a volume (or object storage strategy) for api/data/raw_docs, api/data/embeddings, and the SQLite file if used—ephemeral disks lose indexes and uploads on restart.

Authentication and multi-tenancy

DocSage uses JWT-based authentication with email/password registration and optional Google OAuth.

Backend

  • User model in api/app/models.py: email (unique), hashed password (nullable for OAuth-only), optional oauth_provider/oauth_sub.
  • Auth router at /api/v1/auth/ (api/app/routers/auth.py):
    • POST /register — email + password; returns JWT.
    • POST /login — email + password; returns JWT.
    • GET /me — current user from token.
    • GET /google — redirects to Google consent screen.
    • GET /google/callback — exchanges code, upserts user, redirects to frontend with token in URL hash.
  • Dependencyget_current_user in api/app/deps.py protects all non-auth routes.
  • Tenant isolation: every query in documents, transactions, analytics, anomalies, compare, receipts, export, reports, and chat is filtered by user_id.
  • Per-user upload paths: files are stored under data/raw_docs/user_{id}/.
  • Per-user RAG: FAISS indexes live at data/embeddings/user_{id}/.
  • Chat sessions API at /api/v1/chat/sessions (CRUD scoped to current user).

Frontend

  • AuthProvider in web/src/contexts/auth.tsx: stores JWT in localStorage, exposes login, register, logout, setTokenFromOAuth.
  • Bearer token added to all API requests via web/src/lib/api.ts.
  • Route protection in ShellLayout: unauthenticated users are redirected to /login; public routes: /, /login, /register, /auth/callback.
  • Login/Register pages: Google sign-in is shown only when the API reports OAuth is enabled (GET /auth/config) or NEXT_PUBLIC_GOOGLE_OAUTH_ENABLED=true.
  • Chat sessions sync to the server API when authenticated; fall back to localStorage when offline.

Configuration

VariablePurpose
JWT_SECRETSecret for HS256 token signing (change in production).
JWT_EXPIRE_MINUTESToken validity (default 7 days).
GOOGLE_OAUTH_CLIENT_IDGoogle Cloud console client ID (optional).
GOOGLE_OAUTH_CLIENT_SECRETMatching secret.
GOOGLE_OAUTH_REDIRECT_URIMust match console; default http://localhost:8000/api/v1/auth/google/callback.
FRONTEND_URLWhere the OAuth callback redirects with the token hash fragment.

Public GET /api/v1/auth/config (no auth): returns { "google_oauth_enabled": boolean } so the web UI can hide “Continue with Google” when OAuth is not configured on the server.

Migration

Run python scripts/migrate_database.py from api/ to add user_id columns to existing tables and create users / chat_sessions tables. Existing rows without a user_id are hidden from authenticated queries until backfilled.


Limitations and extension points

AreaLimitationPossible extension
FAISSIndexFlatL2 is linear; slow at very large NIVF / HNSW, or managed vector DB (Pinecone, pgvector, …).
Index updatesadd_documents rebuilds whole indexIncremental add, background jobs, versioning.
Schemacreate_all only; no Alembic in treeMigrations for production schema evolution.
Chathistory trimmed to 20 turns with content (chat.py)Configurable cap, thread storage, or rolling summary.
OCRHost must have Tesseract unless using Docker imageCloud OCR APIs, better layout models.
Compare routesMounted at /documents/... alongside document CRUDEnsure route ordering in OpenAPI matches FastAPI resolution for edge IDs.
SecretsNever commit real GOOGLE_API_KEY; rotate if leakedSecret manager, .env gitignored (already).

This README is the authoritative high-level map of the codebase; for line-level behavior, follow the links into api/app/ and web/src/.

About

AI-powered Intelligent Document Processing (IDP) system with RAG, anomaly detection, and natural language insights. Local, zero-cost alternative to AWS Textract + Bedrock.

Topics

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages