ARGUS is a production-shaped fleet telemetry platform: devices stream into a Kafka-compatible bus, Ray and Flink harden the data path, Iceberg + Dagster form the lakehouse spine, and drift / OPA incidents / OpenTelemetry close the loop for operators — with a read-only AI copilot for investigation. Clone it, docker compose up, and you get the same contracts and images that target EKS via Terraform, Helm, and Argo CD.
Docs:Getting Started · Demo Script (5 min) · Case Study · Known gaps and what I'd build next · ADRs · CHANGELOG v1.0.0
flowchart LR
Sim["simulator / SDKs"] --> Kafka["Redpanda / MSK"]
Kafka --> Ray["Ray ingest"]
Ray --> QA["stream-processor QA"]
QA --> Iceberg["Iceberg + Trino"]
QA --> Drift["drift-monitor"]
Iceberg --> Dagster["Dagster / MLflow"]
Drift --> IE["incident-engine + OPA"]
IE --> GW["api-gateway"]
Iceberg --> GW
GW --> UI["dashboard"]
GW --> Copilot["ai-copilot"]
IE --> OTel["Prometheus / Grafana / Loki / Jaeger"]
simulator/SDKs → telemetry.raw → Ray → normalized → QA ─┬→ validated → Iceberg / drift
├→ quarantine → Iceberg DLQ
└→ qa_metrics → incidents → gateway → UI
Full design: ARCHITECTURE.md · docs/architecture.md
git clone https://github.com/hamidmatiny/Argus.git &&cd Argus
cp .env.example .env
echo"NEXTAUTH_SECRET=$(openssl rand -base64 32)">> .env
echo"NEXTAUTH_URL=http://localhost:3002">> .env
echo"AUTH_DEMO_OFFLINE=true">> .env
docker compose up -d --build| Open | URL |
|---|---|
| Dashboard | http://localhost:3002 — login operator / operator |
| Grafana | http://localhost:3001 — admin / argus |
| Gateway health | http://localhost:8099/health |
| Redpanda Console | http://localhost:8087 |
curl -s http://localhost:8099/v1/ping
curl -s -X POST http://localhost:8099/v1/telemetry/query \
-H 'Content-Type: application/json' -H 'X-API-Key: demo-viewer' \
-d '{"sql":"SELECT vehicle_id FROM telemetry LIMIT 5","limit":5}'Tear down: docker compose down -v
Dashboard — Overview(placeholder)docs/assets/screenshots/dashboard-overview.png — TODO: capture Overview with live throughput after login.
Dashboard — Incidents(placeholder)docs/assets/screenshots/dashboard-incidents.png — TODO: open incidents + breaker state.
Grafana — SLOs(placeholder)docs/assets/screenshots/grafana-slos.png — TODO: QA pass ratio / gateway latency panels.
Also see dashboard/docs/screenshots/*.png.txt markers.
| Component | Role | README |
|---|---|---|
shared/ | Contracts (Avro / Proto / JSON Schema) | README |
ingestion/ | Simulator + Ray normalize | README |
stream-processor/ | Streaming QA gate | README |
drift-monitor/ | KS + Evidently drift | README |
lakehouse/ | Iceberg writers + Trino | README |
orchestration/ | Dagster + MLflow | README |
simulation/ | Scenario synthetic sensors (Sources→Transforms→Sinks) | README |
incident-engine/ | OPA + circuit breakers | README |
api-gateway/ | Authn/z edge API | README |
observability/ | Metrics, logs, traces, alerts | README |
dashboard/ | Next.js operator UI | README |
ai-copilot/ | Read-only RAG assistant | README |
sdk/python/ | argus-sdk | README |
sdk/typescript/ | @argus/sdk | README |
cli/ | argusctl | README |
infra/ | Terraform · Helm · Argo CD | README |
tests/e2e/ | Smoke · load · chaos | README |
docs/ | MkDocs site · ADRs · demo | README |
| Category | Technologies |
|---|---|
| Streaming | Apache Kafka API, Redpanda, Amazon MSK, Ray, Apache Flink / PyFlink |
| Storage | Apache Iceberg, MinIO, Amazon S3, AWS Glue, Trino, Parquet |
| Orchestration | Dagster, Feast (optional); AWS Lambda + Step Functions + EventBridge + SQS (serverless ETL demo — not the production Kafka path; see ADR 007) |
| ML | MLflow, Evidently, SciPy KS tests, embeddings (hash / OpenAI); scenario synthetic sensors (classical renderer proxies — see simulation/) |
| Observability | OpenTelemetry, Prometheus, Grafana, Loki, Promtail, Jaeger, Alertmanager |
| Security | Keycloak (OIDC), OPA/Rego, API keys + RBAC, Trivy, Syft SBOM, Semgrep |
| Frontend | Next.js (App Router), TypeScript, Tailwind, NextAuth, Recharts |
| Infra | Docker Compose, Terraform, Helm, Argo CD, Amazon EKS, IRSA |
| AI | Qdrant, tool-calling LLM agents (OpenAI / Anthropic / mock), RAG runbooks |
| Languages | Python 3.12, Go 1.22, TypeScript / Node 22 |
| Quality | pytest, Go test, Vitest, Playwright, k6, GitHub Actions |
CI coverage gate: ≥ 65% on gated packages (see CONTRIBUTING.md).
pip install -r docs/requirements.txt
mkdocs serve # http://127.0.0.1:8000Apache License 2.0 — see LICENSE.