CLF09 — Multi-Format Malware Detection Platform

Production-ready AI-powered malware detection system across 4 file formats, deployed on AWS serverless infrastructure.

Live Demo · API Reference · Quick Start

---

Overview

CLF09 is a multi-format malware detection platform that classifies threats across Windows PE executables, PDFs, URLs, and Office/VBA documents. Each detector runs as an independent AWS Lambda microservice, enabling isolated scaling and sub-3-second inference in production.

The system combines classical feature engineering (binary/assembly analysis, TF-IDF, structural features) with gradient boosting models optimized via Optuna hyperparameter search — achieving up to 99.82% accuracy on benchmark datasets.

---

Key Results

Detector	Algorithm	Dataset	Samples	Accuracy	Latency
Windows PE	LightGBM + Optuna	Microsoft BIG 2015 (Kaggle)	21,000	99.82%	~1.73s
PDF	LightGBM	CIC-Evasive-PDFMal2022	10,023	99.35%	~1.26s
URL	XGBoost	ISCX-URL-2016	651,191	98.23%	~0.50s
Office/VBA	XGBoost + Optuna	Custom (VirusTotal, Malware Bazaar)	18,538	96.00%	~2.10s

---

Detectors

Windows PE — 9-Class Malware Family Classification

• 1,615 features extracted from raw bytes and assembly code
• Detects : Ramnit, Lollipop, Kelihos, Vundo, Simda, Tracur, Obfuscator, Gatak + Benign
• LightGBM optimized with Optuna (50 trials)
• F1-Score : 99.82% · Precision : 99.81% · Recall : 99.83%

PDF — JavaScript & XFA Exploit Detection

• 23 structural features : JavaScript presence, XFA forms, encryption, object counts
• Binary classification : malicious vs. benign

URL — 4-Class Threat Classification

• 5,015 features : 15 numerical + 5,000 TF-IDF tokens
• Classes : Benign · Defacement · Phishing · Malware
• Trained on 651K URLs from the ISCX-URL-2016 benchmark

Office/VBA — Macro Malware Detection

• 9 VBA-specific features : macro presence, auto-execution keywords, suspicious API calls
• Covers Word (.doc, .docx) and Excel (.xls, .xlsx)

---

Architecture

┌──────────────────────────────────────────────┐
│ Client Layer │
│ Browser · Mobile App · API Clients │
└──────────────────┬───────────────────────────┘
│
┌────────▼────────┐
│ API Gateway │
│ (REST API) │
└────────┬────────┘
│
┌─────────────┼─────────────┬─────────────┐
│ │ │ │
┌────▼────┐ ┌─────▼────┐ ┌────▼────┐ ┌─────▼────┐
│ Lambda │ │ Lambda │ │ Lambda │ │ Lambda │
│ Windows │ │ PDF │ │ URL │ │ Office │
│ 2048 MB │ │ 2048 MB │ │ 2048 MB │ │ 2048 MB │
└────┬────┘ └─────┬────┘ └────┬────┘ └─────┬────┘
└─────────────┴────────────┴──────────────┘
│
┌────────▼────────┐
│ Amazon S3 │
│ (Model Store) │
└─────────────────┘

---

Quick Start

git clone https://github.com/Bassongo/malware_classification.git
cd malware_classification
pip install -r requirements.txt

Prerequisites : Python 3.11+ · Docker · AWS CLI configured

---

API Reference

Base URL : https://63d3m7xcjf.execute-api.us-east-1.amazonaws.com/prod

Endpoint	Method	Input
/detect/windows	POST	Base64-encoded .exe
/detect/pdf	POST	Base64-encoded .pdf
/detect/url	POST	URL string
/detect/office	POST	Base64-encoded .doc/.xls

Example — URL Detection

curl -X POST https://63d3m7xcjf.execute-api.us-east-1.amazonaws.com/prod/detect/url \
-H "Content-Type: application/json" \
-d '{"url": "http://suspicious-site.com/payload"}'

Response

{
"prediction": "malware",
"family": "trojan-downloader",
"confidence": 0.97
}

Example — Windows PE (Python)

importrequests, base64withopen("sample.exe", "rb") asf:
payload=base64.b64encode(f.read()).decode()
response=requests.post(
"https://63d3m7xcjf.execute-api.us-east-1.amazonaws.com/prod/detect/windows",
json={"file_content": payload, "file_name": "sample.exe"}
)
result=response.json()
print(f"Family: {result['family']} | Confidence: {result['confidence']:.2%}")

---

Project Structure

malware_classification/
├── notebooks/
│ ├── windows/ # 7 notebooks — feature engineering & LightGBM training
│ ├── pdf/ # 3 notebooks — structural feature extraction
│ ├── urls/ # 2 notebooks — TF-IDF pipeline & XGBoost
│ └── office/ # 3 notebooks — VBA analysis & classification
├── lambda_deploy/
│ ├── lambda_deploy_urls/ # URL Lambda microservice
│ ├── lambda_deploy_office/ # Office Lambda microservice
│ └── clf09-webapp/ # React 18 + Vite + TailwindCSS frontend
├── requirements.txt
├── .gitignore
├── LICENSE
└── README.md

---

Datasets

Dataset	Source	Samples	Link
Microsoft Malware Classification (BIG 2015)	Kaggle	21,000	Kaggle
CIC-Evasive-PDFMal2022	Canadian Institute for Cybersecurity	10,023	UNB
ISCX-URL-2016	Canadian Institute for Cybersecurity	651,191	UNB
Office/VBA Custom	VirusTotal · Malware Bazaar	18,538	Custom collection

---

Tech Stack

Layer	Technologies
ML	LightGBM · XGBoost · scikit-learn · Optuna
Feature Engineering	NumPy · Pandas · TF-IDF
Cloud	AWS Lambda · API Gateway · S3 · ECR
Frontend	React 18 · Vite · TailwindCSS
DevOps	Docker

---

Limitations & Future Work

• Evasion robustness : adversarial PE files (packed, obfuscated) may fool byte-level features — adding dynamic analysis (sandbox traces) would improve robustness
• Zero-day detection : current models are trained on known families; anomaly detection (isolation forest, autoencoders) could flag novel malware
• Relevance to AI Safety : the adversarial threat model here mirrors alignment challenges — a system optimized against a fixed classifier will find exploits; this motivates ongoing model updates and interpretability research
• Interpretability : adding SHAP explanations per prediction would make the system more auditable and trustworthy

---

Team

CLF09 — AS3, ENSAE Dakar (2024–2025)

Name	Role
Marc MARE	ML Engineering · AWS Deployment
Fatou Soumaya WADE	Feature Engineering · PDF Detector
Gilbert OUMSAORE	URL & Office Detectors
Ndèye Aissatou CISSE	Data Pipeline · Evaluation

Academic supervisor : Mme Fatou SALL

---

Author

Marc MARE — Statistics & ML Engineer
ENSAE Dakar | MSc SEP, University of Reims (2026)

---

License

MIT License — see LICENSE for details.