LLaMA 2 Fine-Tuning with LoRA

This project demonstrates fine-tuning LLaMA 2 language models using Parameter-Efficient Fine-Tuning (PEFT) with LoRA (Low-Rank Adaptation) on macOS with Apple Silicon.

Features

• Fine-tune TinyLLaMA model using LoRA
• Optimized for Apple Silicon (MPS backend support)
• Uses the Alpaca instruction format
• Minimal dataset for quick experimentation
• Efficient training with gradient accumulation

Requirements

• Python 3.11 or higher
• macOS with Apple Silicon (M1/M2/M3)
• Homebrew (for Python installation)

Setup

1. Install Python 3.11

brew install python@3.11

2. Create and Activate Virtual Environment

python3.11 -m venv llm-ft
source llm-ft/bin/activate

3. Install Dependencies

pip install -r requirements.txt

Deactivating the Environment

When you're done working, deactivate the virtual environment:

deactivate

Or if using conda:

conda deactivate

Project Structure

llama2-fine-tuning/
├── llama2-ft.ipynb          # Main training notebook
├── tiny_alpaca.json         # Training dataset (Alpaca format)
├── requirements.txt         # Python dependencies
├── tinyllama/              # Pre-trained TinyLLaMA model
└── llm-ft/                 # Virtual environment

Usage

1. Start Jupyter:

   jupyter notebook llama2-ft.ipynb

2. Run the cells sequentially to:

   • Load the TinyLLaMA model
   • Prepare the instruction dataset
   • Configure LoRA parameters
   • Fine-tune the model
   • Test the fine-tuned model

Training Configuration

• Model: TinyLLaMA
• Method: LoRA (Low-Rank Adaptation)
• LoRA rank: 8
• Target modules: q_proj, v_proj
• Batch size: 1
• Gradient accumulation: 2 steps
• Max sequence length: 512 tokens
• Training steps: 20 (for quick testing)

Dependencies

• torch
• transformers
• datasets
• peft
• accelerate
• trl
• tqdm
• sentencepiece
• ipykernel
• ipywidgets

Notes

• This project uses fp32 precision for compatibility with Apple Silicon
• MPS (Metal Performance Shaders) backend is used for GPU acceleration
• The training dataset is kept small for demonstration purposes
• Adjust training parameters in the notebook for production use

License

Please refer to the LICENSE.txt files in the model directories for licensing information.