ServeStack — ship a model to production, the full DevOps way

ServeStack takes a trained machine-learning model and carries it to a managed Kubernetes cluster with everything a real production service needs: a containerized FastAPI server, health/readiness probes, autoscaling, infrastructure-as-code, a CI/CD pipeline, and golden-signal monitoring.

The served model is intentionally simple (a scikit-learn classifier on a bundled dataset). This project is about the serving platform, not the model — swap in any scikit-learn model by replacing model/train_model.py and the app/schemas.py contract.

Companion to RidePulse (which builds a model). ServeStack ships one.

What's in the box

Concern	Tech
API	FastAPI + Pydantic (validated /predict, /health, /ready, /metrics)
Container	multi-stage Docker, non-root, HEALTHCHECK
Local stack	Docker Compose: API + Prometheus + Grafana
Orchestration	Kubernetes manifests + kustomize overlays + Helm chart
Infrastructure	Terraform for AWS VPC + EKS + ECR
CI/CD	Jenkins pipeline + GitHub Actions
Observability	Prometheus metrics, alert rules, Grafana dashboard
Docs	architecture, ADRs, deploy + on-call runbooks, SLOs

Quickstart (local)

make setup # venv + dev deps
make train # train the demo model -> model/model.joblib
make test# API contract tests (pytest + httpx)
make up # API :8000, Prometheus :9090, Grafana :3000 (docker compose)
make smoke # hit the running API

Example:

curl -X POST localhost:8000/predict -H 'content-type: application/json' \
-d '{"sepal_length":5.1,"sepal_width":3.5,"petal_length":1.4,"petal_width":0.2}'# {"prediction":"setosa","probabilities":{...},"model_version":"1.0.0"}

Grafana at localhost:3000 (anon access on) shows request rate, error ratio, and latency p50/p95/p99.

Project structure

servestack/
├── app/ # FastAPI service (main, model, schemas)
├── model/ # training script + baked model artifact
├── tests/ # pytest API contract tests
├── docker/Dockerfile # multi-stage, non-root runtime
├── docker-compose.yml # api + prometheus + grafana
├── monitoring/ # prometheus.yml, alert rules, grafana provisioning
├── k8s/ # Kubernetes manifests + kustomize prod overlay
├── helm/servestack/ # Helm chart (templated equivalent)
├── terraform/ # VPC + EKS + ECR (registry modules)
├── jenkins/Jenkinsfile # CI/CD pipeline
├── .github/workflows/ # GitHub Actions mirror
├── docs/ # architecture, ADRs, runbooks, SLOs
├── Makefile
└── pyproject.toml

Deploy to production

Full path in docs/deployment.md: terraform apply provisions ECR + EKS; CI builds and pushes the image by git SHA; Helm/kustomize rolls it out with probes + HPA; Prometheus scrapes it via a ServiceMonitor. (Deployable, but not stood up on a live cluster here — that needs an AWS account.)

Architecture

See docs/architecture.md for the system diagram, request flow, and design decisions.

License

MIT — see LICENSE.