Binary classification in R, and a study of what class imbalance does to a model that ignores it.
Written in 2020 during my MSc Data Analytics at London Metropolitan University. The ensemble work here fed into my thesis, which was awarded best master's thesis.
Most real-world classification data is imbalanced, and a model trained without accounting for it will happily report 95% accuracy while never once predicting the class you actually care about.
These reports work through what goes wrong, when sampling is the right response, and which sampling method to reach for.
| Report | What it covers |
|---|---|
| Importance of solving data imbalance | A demonstration of the consequences of ignoring imbalance, and how performance changes once sampling is applied |
| Machine learning ensembles | In-depth treatment of imbalance, ensemble methods, and the performance metrics that stay honest when classes are skewed |
The metric point matters more than the sampling one: under imbalance, accuracy is actively misleading, and precision, recall and AUC are what tell you whether the model works.
Logistic regression and decision trees — an end-to-end binary classification on the German Credit dataset: preparation, cleaning, feature engineering, model fitting, and checking model adequacy rather than stopping at a score.
| File | Contents |
|---|---|
Predictive modeling.R | Modelling script |
Dealing with data imbalance.Rmd | Source for the imbalance report |
LogisticRegression & Decision tree.Rmd | Source for the modelling walkthrough |
R · caret · ROSE/DMwR sampling · rpart · ggplot2 · R Markdown
Rscriptinstall.R# caret, ROSE, DMwR, rpart, pROC, PRROC, MLmetrics, tidyverse, ...LogisticRegression & Decision tree.Rmd downloads the German Credit dataset directly from the UCI machine learning repository at run time, so the main modelling walkthrough is fully reproducible with no manual setup.
Dealing with data imbalance.Rmd reads EnquiriesClean.csv, which is fabricated travel-enquiry data and is not committed. The written report carries its results, and the technique it demonstrates transfers to any imbalanced binary target.
For current work, see rag-eval-harness and medallion-duckdb.