A self-hosted API gateway with rate limiting, request logging, and usage analytics.
SentryFlow sits in front of your APIs as a gateway: it authenticates requests by API key, enforces per-endpoint rate limits, and streams every request through Kafka into ClickHouse so usage patterns, error rates, and latency are queryable after the fact instead of only visible in scattered logs.
- API key management: issue, scope, and revoke keys per user.
- Rate limiting: sliding-window and token-bucket algorithms, configurable per endpoint.
- Request logging: every request is logged with timing, status, and caller identity for auditing and debugging.
- Streaming analytics: request logs are published to Kafka and batch- consumed into ClickHouse, so the dashboard queries pre-aggregated metrics instead of scanning raw logs.
- Dashboard: usage charts, a per-user view, a live rate-limit monitor, and a logs explorer, all in one React app.
flowchart LR
CLIENT[API caller] -->|API key| BACKEND
subgraph BACKEND["Backend (FastAPI)"]
AUTH[Auth middleware]
RATE[Rate limiter\nsliding window / token bucket]
LOG[Logging middleware]
end
BACKEND --> UPSTREAM[Your API]
BACKEND -->|request events| KAFKA[(Kafka)]
KAFKA --> AGG[Aggregator\nbatch consumer]
AGG --> CH[(ClickHouse)]
DASH[React Dashboard] -->|metrics queries| CH
DASH -->|key mgmt, config| BACKEND
Backend: FastAPI, SQLAlchemy, Redis (rate-limit counters), Kafka producer. Aggregator: a standalone Kafka consumer that batches request events into ClickHouse. Frontend: React + Chart.js + Tailwind, talking to both the backend (auth, key management, rate-limit config) and ClickHouse-backed analytics endpoints (dashboards, logs explorer).
git clone https://github.com/Heet852003/SentryFlow.git
cd SentryFlow
docker compose up -d# Backendcd backend
python -m venv venv &&source venv/bin/activate # Windows: venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env
uvicorn main:app --reload
# Aggregator (separate terminal)cd aggregator
pip install -r requirements.txt
cp .env.example .env
python batch_consumer.py
# Frontend (separate terminal)cd frontend
npm install
npm startPrerequisites: Python 3.9+, Node 16+, PostgreSQL, Redis, Kafka, ClickHouse
(or just use docker compose up -d, which brings up all of them).
- Register/log in, then create an API key from the API Keys page. It's shown once, store it client-side.
- Configure per-endpoint rate limits from the Rate Limit Monitor: pick sliding-window or token-bucket, set the limit and window.
- Watch traffic land in the Dashboard, drill into a specific user from User View, or inspect raw requests in the Logs Explorer.
backend/ FastAPI gateway: auth, rate limiting, request logging
aggregator/ Kafka consumer that batches request logs into ClickHouse
frontend/ React dashboard
docs/ API, deployment, and rate-limiting docs
scripts/ dev/docker startup and mock-data helpers
See docs/api.md, docs/deployment.md,
and docs/rate-limiting.md for details.
cd backend
pytestContributions are welcome. Fork the repo, create a feature branch, and open a pull request. See CONTRIBUTING.md.
MIT. See LICENSE for details.