Commit Risk Predictor

ML-Based Commit Risk Scoring for CI/CD — flags high-risk code commits before they reach production using machine learning on git diff semantics, code churn, and author history.

---

Overview

Most production incidents are traceable to a single commit. This system analyzes every commit in your pipeline — extracting semantic features from diffs, code churn patterns, and contributor history — and outputs a risk score before the merge happens.

Integrates as a webhook into any CI/CD pipeline. High-risk commits get flagged automatically, low-risk commits pass through uninterrupted.

---

Architecture

GitHub Webhook
│
▼
┌─────────────────┐
│ Ingestion Layer │ ← src/ingestion/ Receives and queues commit events
└────────┬────────┘
│
▼
┌─────────────────┐
│ Feature Engine │ ← src/features/ Diff parsing, code churn, author stats
└────────┬────────┘
│
▼
┌─────────────────┐
│ Training Layer │ ← src/training/ Model training, evaluation, versioning
└────────┬────────┘
│
▼
┌─────────────────┐
│ Inference API │ ← src/inference/ REST API for real-time risk scoring
└────────┬────────┘
│
▼
┌─────────────────┐
│ Decision Layer │ ← src/decision/ Thresholding, label assignment, routing
└────────┬────────┘
│
▼
┌─────────────────┐
│ Monitoring │ ← src/monitoring/ Drift detection, score logging, alerts
└─────────────────┘
│
▼
Streamlit Dashboard ← dashboard.py Real-time risk visualization

---

Features

• End-to-end ML pipeline — ingestion → feature engineering → training → inference → monitoring
• Webhook-first design — plugs directly into GitHub, GitLab, or any CI system
• Rich feature extraction — diff semantics, lines added/deleted, file churn, author risk history
• Real-time scoring API — low-latency REST endpoint for inline CI/CD gating
• Streamlit dashboard — live commit risk feed with score trends and model metrics
• Docker Compose — entire stack runs with a single command
• Configurable thresholds — tune LOW / MEDIUM / HIGH risk boundaries per repo

---

Project Structure

Commit-Risk-predictor/
│
├── src/
│ ├── ingestion/ # Commit event collection and preprocessing
│ ├── features/ # Feature extraction from git diffs and metadata
│ ├── training/ # Model training, cross-validation, artifact saving
│ ├── inference/ # REST API for real-time risk prediction
│ ├── decision/ # Risk label assignment and routing logic
│ ├── monitoring/ # Score drift, model health, logging
│ ├── utils/ # Shared helpers and constants
│ └── webhook/ # GitHub webhook handler
│
├── scripts/
│ ├── run_api.py # Start the inference API
│ ├── run_training.py # Trigger model training
│ ├── run_ingestion.py # Run commit ingestion pipeline
│ ├── run_feature_engineering.py # Extract features from raw commits
│ ├── run_labeling.py # Label commits from bug-fix history
│ ├── run_dashboard.py # Launch Streamlit dashboard
│ └── generate_test_data.py # Generate synthetic commits for testing
│
├── config/ # Environment and model configuration
├── data/ # Raw commits, features, and labeled datasets
├── models/ # Saved model artifacts
├── notebooks/ # Exploratory analysis
├── tests/ # Unit and integration tests
├── docs/ # Architecture and API documentation
├── logs/ # Runtime and inference logs
│
├── dashboard.py # Streamlit risk dashboard
├── Dockerfile # API container
├── Dockerfile.dashboard # Dashboard container
├── docker-compose.yml # Full stack orchestration
├── requirements.txt # Core dependencies
├── requirements-api.txt # API-specific dependencies
├── requirements-dashboard.txt
├── render.yaml # Render.com deployment config
└── .env.example # Environment variable template

---

Quick Start

1. Clone and configure

git clone https://github.com/Sahojit/Commit-Risk-predictor.git
cd Commit-Risk-predictor
cp .env.example .env
# Fill in your GitHub token and other config in .env

2. Run with Docker

docker-compose up --build

This starts:

• Inference API on http://localhost:8000
• Streamlit Dashboard on http://localhost:8501

3. Run locally

pip install -r requirements.txt
# Ingest commits from a repo
python scripts/run_ingestion.py
# Engineer features
python scripts/run_feature_engineering.py
# Label commits using bug-fix history (SZZ algorithm)
python scripts/run_labeling.py
# Train the risk model
python scripts/run_training.py
# Start the inference API
python scripts/run_api.py
# Launch the dashboard
python scripts/run_dashboard.py

---

API Usage

Score a commit

POST /predict
Content-Type: application/json
{
"commit_sha": "a3f92c1",
"repo": "owner/repo",
"diff": "...",
"author": "dev@example.com"
}

Response:

{
"commit_sha": "a3f92c1",
"risk_score": 0.87,
"risk_label": "HIGH",
"flagged_files": [
"src/auth/token_handler.py",
"src/db/session.py"
],
"top_features": {
"lines_deleted": 67,
"author_bug_rate": 0.31,
"files_changed": 3,
"churn_ratio": 0.82
}
}

GitHub Webhook

Point your repo's webhook at:

POST https://<your-host>/webhook/github

Every push event is automatically scored and logged to the dashboard.

---

Configuration

Copy .env.example to .env and set:

Variable	Description
GITHUB_TOKEN	GitHub API token for commit fetching
RISK_THRESHOLD_HIGH	Score threshold for HIGH risk (default: 0.75)
RISK_THRESHOLD_MEDIUM	Score threshold for MEDIUM risk (default: 0.45)
MODEL_PATH	Path to saved model artifact
LOG_LEVEL	Logging verbosity (INFO, DEBUG)

---

Dashboard

The Streamlit dashboard shows:

• Live commit risk feed — every scored commit with label and score
• Risk distribution — histogram of scores over time
• Model metrics — precision, recall, AUC-ROC on recent predictions
• High-risk alerts — commits above threshold highlighted for review

python scripts/run_dashboard.py
# or
streamlit run dashboard.py

---

Training Your Own Model

# 1. Ingest commit history from target repo
python scripts/run_ingestion.py --repo owner/repo --limit 5000
# 2. Extract features from raw diffs
python scripts/run_feature_engineering.py
# 3. Auto-label using SZZ (links bug-fix commits to bug-introducing ones)
python scripts/run_labeling.py
# 4. Train and evaluate
python scripts/run_training.py
# Model saved to models/ with evaluation report

---

Testing

# Generate synthetic test data
python scripts/generate_test_data.py
# Run tests
pytest tests/ -v

---

Deployment

The repo includes a render.yaml for one-click deployment to Render:

# Push to main — auto-deploy triggers via render.yaml

Or deploy with Docker to any cloud VM:

docker-compose up -d

---

Tech Stack

Layer	Technology
Language	Python 3.11
ML	scikit-learn, gradient boosting
Feature Extraction	gitpython, unidiff
API	FastAPI
Dashboard	Streamlit
Containerization	Docker, Docker Compose
CI Integration	GitHub Webhooks
Deployment	Render
Logging	Python logging + file rotation

---

Author

Sahojit Karmakar — AI/ML Engineer

GitHub · LinkedIn · Email