Distracted-Drivers

Goal

Fine-tune and apply feature engineering and adapters to Efficientnet for a classification task and examine the classwise accuracy consequences.

Fine tuning

This simply corresponds to retrainig the top layers at a high learning rate, and the body layers at a lower learning rate, such that we maintain the core of the model knowledge, but quickly converge towards the ends to our task specific classifications. This is implemented by a few lines in the main file classifier.py

Feature engineering: Image segmentation

Big picture: At layer L-1, we not only feed it the outputs from L-2, but feed it a larger vector

Old L-1: (classifier): Linear(in_features=1536, out_features=10)

New L-1 (classifier): Linear(in_features=1536 + len(feature_engineering), out_features=10)

The feature engineering we apply is 6 classes from a semantic segmentation (Technically it is a panoptic segmentation, merged to become semantic). The classes are

 person
bottle
cell phone
cup
car
chair

See an image example. These are passed through a simple CNN and flattened to create the feature_engineering vector

Adapters

By placing a layer adjacent to a low-level layer, we can fine-tune low-level features to our task, while only having to store the much smaller adapter layer.

Training

python -m distracted.classifiers [OPTIONS]
Options:
--batch-size INTEGER
--model-name TEXT
--adapters TEXT
--top-lr FLOAT
--top-decay FLOAT
--body-lr FLOAT
--body-decay FLOAT
--gamma FLOAT
--epochs INTEGER
--adapter-weight FLOAT
--checkpoint TEXT

Which model is trained? See classifiers.py:

 if adapters:
setup = adapter_setup(...)
elif body_lr:
setup = finetune_setup(...)
else:
setup = segmentation_setup(...)

Models and metrics are stored via MLFlow.org:

cd ./data && mlflow ui

Final evaluations are performed via gradcam.py. It is recommended running this file using VSCode interactive for an interactive data exploration experience.

Hardware requirements

A batch size of 128 is optimal for 12 GB of VRAM. It is highly recommended to have at least 16GB of RAM, preferably 32GB.

Setup

git clone https://github.com/Ispanicus/Distracted-Drivers
cd Distracted-Drivers
./install_windows.bat

Not tested on mac or linux, but the install script should be similar if not identical.

Download the "distracted driver" dataset https://www.kaggle.com/competitions/state-farm-distracted-driver-detection/data

Unzip and rename to Distracted-Drivers/data/

Then cache panoptic masks (Duration: Hours)

python -m distracted.image_segmentation.py