Transform scattered communications into strategic intelligence
InsightFlow automatically ingests emails and meetings, uses ML-powered topic detection to identify patterns, and generates weekly strategic briefs that help leaders stay ahead of organizational drift.
Key Features:
- 🤖 AI-Optional Design: Works perfectly without AI; Gemini enhancement is optional
- 📊 Smart Topic Detection: HDBSCAN clustering + embeddings find emerging patterns
- 🎯 Rules-First Intelligence: Deterministic metrics with 28-day baseline comparison
- 📝 Evidence-Backed: Full audit trail linking insights to original sources
- 🚀 Production Ready: Docker-containerized, fully tested, zero-config startup
# 1. Clone the repository
git clone https://github.com/yourusername/InsightFlow.git
cd InsightFlow
# 2. Create .env file (uses defaults, no API keys needed)
copy .env.example .env
# 3. Start all services
docker-compose up -d
# 4. Open dashboard# http://localhost:8000/static/index.html# That's it! 🎉What you get:
- ✅ PostgreSQL with pgvector
- ✅ FastAPI backend with dashboard
- ✅ Rules-only mode (no AI required)
- ✅ Manual analysis trigger
- ✅ Beautiful web UI
# 1. Get free Gemini API key# Visit: https://makersuite.google.com/app/apikey# 2. Edit .env file# Set: LLM_ENABLED=true# Set: GEMINI_API_KEY=your-key-here# 3. Restart services
docker-compose restart api
# Now you have AI-enhanced strategic insights! ✨# 1. Create virtual environment
python -m venv .venv
.venv\Scripts\activate # Windowssource .venv/bin/activate # Linux/Mac# 2. Install dependencies
pip install -r requirements.txt
# 3. Set up PostgreSQL
createdb insightflow_db
psql insightflow_db < sql/schema.sql
# 4. Configure .env
copy .env.example .env
# Edit DATABASE_URL if needed# 5. Start API server
uvicorn api.main:app --reload
# 6. Access dashboard# http://localhost:8000/static/index.html- Setup Guide - Detailed installation instructions
- Docker Guide - Container deployment
- Project Documentation - Complete technical spec
- N8N Testing Guide - Workflow integration
┌─────────────────┐
│ Data Sources │ Gmail, Read.ai, etc.
│ (via n8n) │
└────────┬────────┘
│
▼
┌─────────────────┐
│ FastAPI │ /ingest/email, /ingest/meeting
│ Ingestion │
└────────┬────────┘
│
▼
┌─────────────────┐
│ PostgreSQL + │ Vector storage + relational data
│ pgvector │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Weekly Analysis │ Embeddings → Clustering → Rules → LLM (optional)
└────────┬────────┘
│
▼
┌─────────────────┐
│ Dashboard + │ Web UI + Slack delivery
│ Delivery │
└─────────────────┘
Processing Pipeline:
- Ingestion - Validate & store events (emails/meetings)
- Embeddings - Generate 384-dim vectors (MiniLM)
- Clustering - HDBSCAN topic detection
- Metrics - Compute deltas vs 28-day baseline
- Rules - Apply 4 deterministic finding rules
- LLM - Optional Gemini enhancement
- Render - Markdown brief + JSON audit trail
- Deliver - Dashboard + Slack notification
Access at http://localhost:8000/static/index.html
- Overview - Real-time metrics, charts, latest brief preview
- Weekly Briefs - Historical briefs with LLM enhancement indicators
- Trends - Week-over-week visualizations
- Recent Events - Sortable table with source links for traceability
- Manual Trigger - "Run Analysis Now" button (no wait for weekly timer)
- LLM Status - Visual indicator showing AI enhancement status
Edit .env file to configure:
| Variable | Default | Description |
|---|---|---|
LLM_ENABLED | false | Enable/disable AI enhancement |
GEMINI_API_KEY | - | API key for Gemini (optional) |
DATABASE_URL | docker default | PostgreSQL connection string |
SLACK_WEBHOOK_URL | - | Slack delivery (optional) |
CLUSTERING_MIN_SIZE | 3 | Minimum events per topic |
EMERGING_RISK_FREQ_PCT | 30 | Risk detection threshold |
See .env.example for all options.
# Disable AI (rules-only mode)
LLM_ENABLED=false
# Enable AI (requires API key)
LLM_ENABLED=true
GEMINI_API_KEY=your-key-here
# Restart to apply
docker-compose restart apiImport workflows from n8n/workflows/:
gmail-ingestion.json- Email automationreadai-ingestion.json- Meeting transcripts
# Ingest an email
curl -X POST http://localhost:8000/ingest/email \
-H "Content-Type: application/json" \
-d '{ "id": "email_001", "source": "email", "timestamp": "2026-01-08T10:00:00Z", "actor": "alice@company.com", "direction": "inbound", "subject": "Q1 Budget Review", "text": "Attached is the revised budget...", "thread_id": "thread_001", "decision": "deferred", "action_owner": null, "follow_up_required": true, "urgency_score": 7, "sentiment": "unknown", "raw_ref": "https://mail.google.com/..." }'# Run all tests
pytest
# With coverage
pytest --cov=src --cov-report=html
# Specific test suite
pytest tests/unit/
pytest tests/api/
pytest tests/integration/InsightFlow/
├── api/ # FastAPI backend
│ ├── main.py # Ingestion endpoints
│ ├── dashboard.py # Dashboard API + settings
│ └── schemas.py # Pydantic models
├── src/insight/ # Core engine
│ ├── clustering.py # HDBSCAN topic detection
│ ├── llm_client.py # Gemini integration
│ ├── rules.py # Finding generation
│ ├── renderer.py # Markdown + JSON output
│ └── services.py # Metrics computation
├── scripts/ # Automation
│ ├── weekly_run.py # Main orchestration
│ └── export_brief.py # Export utilities
├── static/ # Web dashboard
│ └── index.html # Single-page app
├── sql/ # Database schema
│ └── schema.sql # PostgreSQL + pgvector
├── n8n/ # Workflow templates
├── tests/ # Test suites
└── docker-compose.yml # Container orchestration
# Production deployment
docker-compose -f docker-compose.prod.yml up -d
# Set up weekly automation
sudo cp systemd/insightflow-weekly.* /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable insightflow-weekly.timer
sudo systemctl start insightflow-weekly.timerSee SETUP_GUIDE.md for Hostinger VPS deployment.
- API keys in
.env(gitignored) - Database credentials in environment variables
- No sensitive data sent to LLM (only pre-computed metrics)
- Slack webhooks for secure delivery
- Fork the repository
- Create feature branch (
git checkout -b feature/amazing-feature) - Make changes and add tests
- Run test suite (
pytest) - Commit with conventional commits
- Push and open Pull Request
MIT License - See LICENSE for details
- Clone and run immediately (Docker)
- Add to existing n8n workflows
- Customize rules for your domain
- Self-host on any VPS
- Clean architecture example
- Production-ready FastAPI patterns
- ML pipeline best practices
- Docker deployment templates
- Embedding-based clustering
- LLM-optional design patterns
- Deterministic + AI hybrid systems
- Full audit trail for reproducibility
First Time Setup:
- Start with Docker (zero config)
- Test with manual analysis trigger
- Add sample data via API
- Enable AI when ready
Customization:
- Edit rules in
src/insight/rules.py - Adjust thresholds in
.env - Modify dashboard in
static/index.html - Add new metrics in
src/insight/services.py
Troubleshooting:
- Check logs:
docker-compose logs -f api - Test API:
http://localhost:8000/docs - Verify DB:
docker-compose exec db psql -U insightflow - LLM status: Dashboard header shows active/disabled
- Issues: GitHub Issues
- Discussions: GitHub Discussions
- Email: johnsavilesh@gmail.com
Transform your organization's communication into strategic intelligence 🚀