Single-Turn User Utterance Prediction in Dialogue Contexts with QLoRA

This repository targets a conference submission extending the outcome of a bachelor thesis.

QLoRA fine-tuning of open-source instruction-tuned LLMs (1.2B–7B parameters) to predict a single next user utterance from dialogue context. Trained on 6,000 samples from WildChat (open-domain) and Schema-Guided Dialog (task-oriented), evaluated with BERTScore, BLEURT, perplexity, and a blinded human study of 369 samples.

Key Results

Model	BERTScore F1	BLEURT	Perplexity
Qwen-3B (fine-tuned)	0.846	0.384	10.4
Llama-3B (fine-tuned)	0.854	0.393	11.2
OLMo-7B (fine-tuned)	0.798	0.229	9.0
LiquidAI-1.2B (fine-tuned)	0.837	0.373	9.4
GPT-4o-mini (few-shot)	0.847	0.401	—

Project Structure

├── src/ # Modular training & evaluation pipeline
│ ├── main.py # Entry point (full pipeline)
│ ├── config.py # Centralized hyperparameters
│ ├── data.py # Dataset loading & preprocessing
│ ├── model.py # Model loading (QLoRA + 4-bit)
│ ├── train.py # SFTTrainer wrapper
│ ├── evaluate.py # Baseline & fine-tuned evaluation
│ ├── ablation.py # Two-stage ablation study
│ ├── temperature_sweep.py # Temperature sweep experiments
│ └── prompt_baseline.py # GPT-4o-mini prompt baselines
├── scripts/ # Shell scripts for cloud deployment
│ ├── run_all_models.sh # Run pipeline for all 4 models
│ ├── gcloud_run.sh # GCP VM provisioning & execution
│ └── run_docker.sh # Docker-based execution
├── paper/ # IEEE conference paper (LaTeX)
│ ├── IEEE-conference-template-062824.tex
│ ├── reference.bib
│ └── generate_figures.py # Publication-quality figures (tueplots)
├── evaluator/ # Next.js app for blinded human evaluation
├── outputs/ # Model evaluation results & CSVs
├── Dockerfile # Reproducible GPU environment
├── docker-compose.yml # Docker Compose with GPU support
└── requirements.txt # Python dependencies

Reproducing Results

Prerequisites

• NVIDIA GPU with ≥40 GB VRAM (tested on A100-SXM4-40GB)
• Docker with NVIDIA Container Toolkit, or Python 3.11 + CUDA 12.1
• Hugging Face token (HF_TOKEN) for gated models
• (Optional) Weights & Biases API key (WANDB_API_KEY)

Option 1: Docker (recommended)

# Clone and configure
git clone https://github.com/your-org/user_turn_lora.git
cd user_turn_lora
cp .env.example .env # Add HF_TOKEN and WANDB_API_KEY# Build image
docker compose build
# Run full pipeline for a single model
docker compose run userturn-lora --model Qwen/Qwen2.5-3B-Instruct
# Quick test run (small dataset, 1 epoch, no W&B)
docker compose run userturn-lora-test
# Run all 4 models sequentially
docker compose run userturn-lora --model Qwen/Qwen2.5-3B-Instruct
docker compose run userturn-lora --model meta-llama/Llama-3.2-3B-Instruct
docker compose run userturn-lora --model allenai/OLMo-3-7B-Instruct
docker compose run userturn-lora --model LiquidAI/LFM2.5-1.2B-Instruct

Option 2: Local

pip install -r requirements.txt
pip install git+https://github.com/google-research/bleurt.git
# Download BLEURT checkpoint
wget https://storage.googleapis.com/bleurt-oss-21/BLEURT-20.zip
unzip BLEURT-20.zip
export BLEURT_CHECKPOINT=./BLEURT-20
# Run pipeline
python -m src.main --model Qwen/Qwen2.5-3B-Instruct

Pipeline CLI

# Full pipeline (train + evaluate baseline + evaluate fine-tuned)
python -m src.main --model Qwen/Qwen2.5-3B-Instruct
# Skip baseline evaluation
python -m src.main --model Qwen/Qwen2.5-3B-Instruct --skip-baseline
# Evaluate existing adapter only
python -m src.main --model Qwen/Qwen2.5-3B-Instruct --skip-training
# Custom hyperparameters
python -m src.main --model Qwen/Qwen2.5-3B-Instruct --epochs 3 --lr 2e-4 --lora-r 8 --lora-alpha 64
# Disable W&B logging
python -m src.main --no-wandb

Ablation Study

python -m src.ablation --model LiquidAI/LFM2.5-1.2B-Instruct

Temperature Sweep

python -m src.temperature_sweep --model Qwen/Qwen2.5-3B-Instruct

GPT-4o-mini Prompt Baselines

export OPENAI_API_KEY=your-key
python -m src.prompt_baseline

Paper Figures

cd paper
python generate_figures.py

Supported Models

Model	HuggingFace ID	Parameters
Qwen2.5-3B-Instruct	Qwen/Qwen2.5-3B-Instruct	3B
Llama-3.2-3B-Instruct	meta-llama/Llama-3.2-3B-Instruct	3B
OLMo-3-7B-Instruct	allenai/OLMo-3-7B-Instruct	7B
LFM2.5-1.2B-Instruct	LiquidAI/LFM2.5-1.2B-Instruct	1.2B

Any Hugging Face model supporting apply_chat_template can be added by extending SPECIAL_TOKENS in src/config.py.

Optimal LoRA Configuration

Determined via a 56-experiment two-stage ablation on LiquidAI-1.2B:

Parameter	Value
LoRA rank (r)	8
LoRA alpha (α)	64
LoRA dropout	0.0
Learning rate	2e-4
Epochs	3
Warmup ratio	0.1
Target modules	q, k, v, o, gate, up, down

Datasets

Dataset	Domain	Train	Eval	Total
WildChat-1M	Open-domain	3,000	200	3,200
Schema-Guided Dialog	Task-oriented	3,000	200	3,200
Total		6,000	400	6,400

Samples are constructed as (context, target) pairs where context ends with an assistant turn and target is the next user utterance. Filtering: English only, ≥3 turns, seed=42.

Human Evaluation

The evaluator/ directory contains a Next.js application for blinded A/B human evaluation:

cd evaluator && bun install && bun run dev

Environment Notes

• Tested on NVIDIA A100-SXM4-40GB (GCP)
• BLEURT requires TensorFlow 2.15.x (Python 3.11 compatible)
• Flash Attention 2 used where available (pre-built wheel in Dockerfile)