diff --git a/docs/en/training_guides/cpt-comprehensive-tutorial.ipynb b/docs/en/training_guides/cpt-comprehensive-tutorial.ipynb new file mode 100644 index 0000000..515a9dc --- /dev/null +++ b/docs/en/training_guides/cpt-comprehensive-tutorial.ipynb @@ -0,0 +1,338 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Comprehensive Continued Pre-training (CPT) Tutorial\n", + "\n", + "This notebook covers **continued pre-training (CPT)** \u2014 also called *continual* or\n", + "*domain-adaptive* pre-training \u2014 with the\n", + "[`training_hub`](https://github.com/Red-Hat-AI-Innovation-Team/training_hub) library on\n", + "Alauda AI.\n", + "\n", + "CPT keeps the original **next-token prediction** objective but runs it over a **raw-text\n", + "corpus** from your domain (medical notes, legal contracts, code, a new language, ...). It\n", + "teaches the model new *knowledge and vocabulary* before any instruction tuning, unlike SFT\n", + "which teaches *behavior* from chat data.\n", + "\n", + "`training_hub` runs CPT through the same `sft(...)` entrypoint with **`is_pretraining=True`**:\n", + "the loss is computed over *all* tokens (no assistant-only masking) and documents are packed\n", + "into fixed-size `block_size` windows." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## When to use CPT\n", + "\n", + "A typical domain-adaptation pipeline is **CPT -> SFT -> alignment**:\n", + "\n", + "1. **CPT** on raw domain text \u2014 inject knowledge / vocabulary.\n", + "2. **SFT** (or **OSFT**) on chat data \u2014 teach the model to follow instructions.\n", + "3. Optional preference/RL alignment.\n", + "\n", + "| | CPT | SFT | OSFT |\n", + "|---|---|---|---|\n", + "| Data | raw text | chat (instruction/response) | chat |\n", + "| Objective | next-token over all tokens | next-token over assistant turns | next-token, orthogonal subspace |\n", + "| Teaches | knowledge / vocabulary | behavior | behavior, forgetting-resistant |\n", + "\n", + "CPT can cause **catastrophic forgetting** of general ability. Mitigate by mixing in some\n", + "general-domain text, keeping the learning rate low, and running a follow-up SFT pass. If\n", + "forgetting is the main concern, consider **OSFT** instead (see\n", + "[Training Hub fine-tuning](../training-hub-fine-tuning.mdx))." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import json\n", + "import os\n", + "\n", + "# CPT trains on RAW TEXT: one document per line under a \"text\" field.\n", + "data_path = \"./test_cpt_data.jsonl\"\n", + "if not os.path.exists(data_path):\n", + " print(f\"Creating dummy raw-text corpus at {data_path}\")\n", + " docs = [\n", + " \"Alauda AI is an MLOps platform that runs fine-tuning and inference workloads on Kubernetes.\",\n", + " \"Continued pre-training adapts a base language model to a new domain using unlabeled text.\",\n", + " \"The training_hub library wraps SFT, OSFT, LoRA, QLoRA and continued pre-training behind one interface.\",\n", + " \"Kubeflow Trainer v2 submits distributed TrainJobs that schedule onto GPU or NPU nodes.\",\n", + " ] * 8\n", + " with open(data_path, \"w\") as f:\n", + " for d in docs:\n", + " f.write(json.dumps({\"text\": d}) + \"\\n\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Data format requirements\n", + "\n", + "CPT data is **raw text**, not chat turns. Each JSONL line is a document under a single\n", + "text column (default `\"text\"`):\n", + "\n", + "```json\n", + "{\"text\": \"A paragraph or document of raw domain text ...\"}\n", + "{\"text\": \"Another document ...\"}\n", + "```\n", + "\n", + "`training_hub` concatenates and chunks the documents into fixed-length `block_size`\n", + "windows and trains next-token prediction over every token (full unmasking).\n", + "\n", + "- `document_column_name` \u2014 the JSONL field holding the text (default `\"text\"`).\n", + "- `block_size` \u2014 the packed context-window length (e.g. 1024-4096)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Model configuration examples\n", + "\n", + "CPT typically starts from a **base** (not already instruction-tuned) checkpoint. Provide a\n", + "HuggingFace name or a local path; pre-download to a local directory on air-gapped clusters." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# =============================================================================\n", + "# MODEL CONFIGURATION EXAMPLES\n", + "# =============================================================================\n", + "\n", + "# Example 1: Qwen 3 0.6B Base (tiny - good for a smoke test)\n", + "qwen_small_example = {\n", + " \"model_name\": \"Qwen 3 0.6B Base\",\n", + " \"model_path\": \"/opt/app-root/src/Qwen3-0.6B-Base\",\n", + " \"example_block_size\": 1024,\n", + " \"example_batch_size\": 8,\n", + " \"example_learning_rate\": 5e-6,\n", + "}\n", + "\n", + "# Example 2: Qwen 2.5 7B (base) - domain adaptation target\n", + "qwen_7b_example = {\n", + " \"model_name\": \"Qwen 2.5 7B\",\n", + " \"model_path\": \"Qwen/Qwen2.5-7B\",\n", + " \"example_block_size\": 4096,\n", + " \"example_batch_size\": 128,\n", + " \"example_learning_rate\": 5e-6,\n", + "}\n", + "\n", + "# =============================================================================\n", + "# SELECT YOUR CONFIGURATION\n", + "# =============================================================================\n", + "selected_example = qwen_small_example\n", + "print(f\"Selected model: {selected_example['model_name']} ({selected_example['model_path']})\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## CPT parameter reference\n", + "\n", + "CPT reuses every `sft(...)` parameter and adds three pre-training controls:\n", + "\n", + "- `is_pretraining=True` \u2014 switch the data path to raw-text pre-training (no chat masking).\n", + "- `block_size` \u2014 packed sequence length for next-token prediction.\n", + "- `document_column_name` \u2014 JSONL field holding the raw text.\n", + "\n", + "Use a **low learning rate** (CPT is sensitive \u2014 `1e-6`-`1e-5`) to limit forgetting." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# =============================================================================\n", + "# COMPLETE CPT PARAMETER CONFIGURATION\n", + "# =============================================================================\n", + "from datetime import datetime\n", + "\n", + "experiment_name = \"cpt_comprehensive_example\"\n", + "timestamp = datetime.now().strftime(\"%Y%m%d_%H%M%S\")\n", + "full_experiment_name = f\"{experiment_name}_{timestamp}\"\n", + "\n", + "# ---- Required ----\n", + "model_path = selected_example[\"model_path\"]\n", + "data_path = \"./test_cpt_data.jsonl\"\n", + "ckpt_output_dir = f\"checkpoints/{full_experiment_name}\"\n", + "\n", + "# ---- Continued pre-training controls ----\n", + "is_pretraining = True\n", + "block_size = selected_example[\"example_block_size\"]\n", + "document_column_name = \"text\"\n", + "\n", + "# ---- Core training ----\n", + "num_epochs = 1\n", + "effective_batch_size = selected_example[\"example_batch_size\"]\n", + "learning_rate = selected_example[\"example_learning_rate\"] # keep low to limit forgetting\n", + "max_seq_len = block_size\n", + "max_tokens_per_gpu = 4096 # per-GPU token budget; auto micro-batch sizing\n", + "warmup_steps = 10\n", + "\n", + "print(\"CPT configuration:\")\n", + "print(f\" pretraining: is_pretraining={is_pretraining}, block_size={block_size}, column='{document_column_name}'\")\n", + "print(f\" train: epochs={num_epochs}, ebs={effective_batch_size}, lr={learning_rate}, max_tokens_per_gpu={max_tokens_per_gpu}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Distributed configuration\n", + "\n", + "CPT uses the same distributed topology knobs as SFT. Raw-text corpora are usually large,\n", + "so multi-GPU / multi-node is common \u2014 set `nproc_per_node` to the GPU count per node." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "nproc_per_node = 1 # raise to the per-node GPU count for real corpora\n", + "nnodes = 1\n", + "node_rank = 0\n", + "rdzv_id = 200\n", + "rdzv_endpoint = \"127.0.0.1:29500\"\n", + "print(f\"distributed: nproc_per_node={nproc_per_node}, nnodes={nnodes}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Execute training\n", + "\n", + "The final cell calls `training_hub.sft(..., is_pretraining=True)` \u2014 continued pre-training\n", + "over the raw-text corpus." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from training_hub import sft\n", + "\n", + "result = sft(\n", + " # --- required ---\n", + " model_path=model_path,\n", + " data_path=data_path,\n", + " ckpt_output_dir=ckpt_output_dir,\n", + " # --- continued pre-training ---\n", + " is_pretraining=is_pretraining,\n", + " block_size=block_size,\n", + " document_column_name=document_column_name,\n", + " # --- core training ---\n", + " num_epochs=num_epochs,\n", + " effective_batch_size=effective_batch_size,\n", + " learning_rate=learning_rate,\n", + " max_seq_len=max_seq_len,\n", + " max_tokens_per_gpu=max_tokens_per_gpu,\n", + " warmup_steps=warmup_steps,\n", + " checkpoint_at_epoch=True,\n", + " # --- distributed ---\n", + " nproc_per_node=nproc_per_node,\n", + " nnodes=nnodes,\n", + " node_rank=node_rank,\n", + " rdzv_id=rdzv_id,\n", + " rdzv_endpoint=rdzv_endpoint,\n", + ")\n", + "print(f\"CPT finished: {result!r}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Post-training analysis\n", + "\n", + "CPT writes a **full checkpoint** (the whole model is updated). The natural next step is an\n", + "SFT / OSFT pass on instruction data so the domain-adapted model can follow instructions." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "\n", + "hf_dir = os.path.join(ckpt_output_dir, \"hf_format\")\n", + "print(f\"Checkpoint dir: {hf_dir}\")\n", + "if os.path.isdir(hf_dir):\n", + " for ckpt in sorted(os.listdir(hf_dir)):\n", + " print(\" checkpoint:\", ckpt)\n", + "\n", + "print(\"\\nNext step: run sft() / osft() on instruction data, pointing model_path at this checkpoint.\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Parameter reference summary\n", + "\n", + "### CPT-specific parameters\n", + "\n", + "| Parameter | Description | Typical |\n", + "|---|---|---|\n", + "| `is_pretraining` | Switch to raw-text pre-training (no chat masking) | `True` |\n", + "| `block_size` | Packed context-window length | 1024-4096 |\n", + "| `document_column_name` | JSONL field with raw text | `\"text\"` |\n", + "\n", + "### Common training parameters\n", + "\n", + "| Parameter | Description |\n", + "|---|---|\n", + "| `model_path`, `data_path`, `ckpt_output_dir` | Required I/O |\n", + "| `num_epochs`, `effective_batch_size`, `learning_rate` | Core training (use a **low** LR) |\n", + "| `max_seq_len`, `max_tokens_per_gpu` | Sequence length / memory budget |\n", + "| `nproc_per_node`, `nnodes`, `node_rank`, `rdzv_id`, `rdzv_endpoint` | Distributed topology |\n", + "\n", + "> **NPU note:** Full-parameter continued pre-training also runs on Huawei Ascend NPU via\n", + "> the `MindSpeed-LLM` runtime (`pretrain_gpt.py`). See\n", + "> [Fine-tune and Pretrain on Ascend NPU](../fine-tune-and-pretrain-llms-on-ascend-npu.mdx)\n", + "> and the `qwen25_pretrain_verify.ipynb` recipe for the Megatron-style pre-training path." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.11" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/en/training_guides/qlora-comprehensive-tutorial.ipynb b/docs/en/training_guides/qlora-comprehensive-tutorial.ipynb new file mode 100644 index 0000000..8093d0f --- /dev/null +++ b/docs/en/training_guides/qlora-comprehensive-tutorial.ipynb @@ -0,0 +1,375 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Comprehensive QLoRA Training Tutorial\n", + "\n", + "This notebook is a hands-on guide to **QLoRA** (Quantized Low-Rank Adaptation)\n", + "fine-tuning with the [`training_hub`](https://github.com/Red-Hat-AI-Innovation-Team/training_hub)\n", + "library on Alauda AI.\n", + "\n", + "QLoRA freezes the base model in **4-bit NormalFloat (NF4)** precision and trains a\n", + "small set of **LoRA adapter** matrices on top. Only the adapters are updated, so a\n", + "7B model that needs ~60 GiB for full SFT fits comfortably on a single 16-24 GiB GPU\n", + "slice \u2014 the whole point of QLoRA (Dettmers et al., 2023, [arXiv:2305.14314](https://arxiv.org/abs/2305.14314)).\n", + "\n", + "`training_hub` exposes this through a single `lora_sft(...)` call: set\n", + "`load_in_4bit=True` plus the LoRA rank/alpha and you have QLoRA.\n", + "\n", + "> Runtime: run this notebook on the `traininghub0.1-cu126-amd64` runtime image, which\n", + "> bundles `trl`, `peft`, and **`bitsandbytes`** \u2014 the 4-bit quantization backend QLoRA needs." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## When to use QLoRA\n", + "\n", + "| | Full SFT | LoRA | **QLoRA** |\n", + "|---|---|---|---|\n", + "| Base weights | trainable (bf16) | frozen (bf16) | **frozen (4-bit NF4)** |\n", + "| Trainable params | 100% | adapters only | adapters only |\n", + "| GPU memory | highest | medium | **lowest** |\n", + "| Output | full checkpoint | base + adapter | base + adapter |\n", + "\n", + "Reach for QLoRA when you are GPU-memory bound \u2014 a large model on a single card or a\n", + "small HAMi vGPU slice. The trade-off is a small quantization-induced quality gap and\n", + "slightly slower steps (de-quantization happens on the fly). For maximum throughput when\n", + "memory is not the constraint, use plain `sft(...)` / `lora_sft(..., load_in_4bit=False)`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import json\n", + "import os\n", + "\n", + "# QLoRA trains on the same chat-style JSONL schema as SFT.\n", + "data_path = \"./test_qlora_data.jsonl\"\n", + "if not os.path.exists(data_path):\n", + " print(f\"Creating dummy dataset at {data_path}\")\n", + " dummy_data = [\n", + " {\"messages\": [\n", + " {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n", + " {\"role\": \"user\", \"content\": \"Hello, how are you?\"},\n", + " {\"role\": \"assistant\", \"content\": \"I am doing well, thank you! How can I help you today?\"},\n", + " ]}\n", + " ] * 10\n", + " with open(data_path, \"w\") as f:\n", + " for d in dummy_data:\n", + " f.write(json.dumps(d) + \"\\n\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Data format requirements\n", + "\n", + "QLoRA uses the **same conversational JSONL** as SFT \u2014 one conversation per line:\n", + "\n", + "```json\n", + "{\"messages\": [{\"role\": \"system\", \"content\": \"...\"}, {\"role\": \"user\", \"content\": \"...\"}, {\"role\": \"assistant\", \"content\": \"...\"}]}\n", + "```\n", + "\n", + "Only `assistant` turns contribute to the loss by default. See the\n", + "[SFT data format](../training-hub-fine-tuning.mdx#data-format) for the full masking rules." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Model configuration examples\n", + "\n", + "QLoRA shines on larger models, but you can adapt any HuggingFace causal-LM checkpoint.\n", + "Provide a HuggingFace name or a local path. For air-gapped clusters, pre-download the\n", + "model (e.g. from a ModelScope mirror) to a local directory and point `model_path` at it." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# =============================================================================\n", + "# MODEL CONFIGURATION EXAMPLES\n", + "# Pick one as a starting point and tune for your hardware.\n", + "# =============================================================================\n", + "\n", + "# Example 1: Qwen 3 0.6B (tiny - good for a single small GPU slice / smoke test)\n", + "qwen_small_example = {\n", + " \"model_name\": \"Qwen 3 0.6B\",\n", + " \"model_path\": \"/opt/app-root/src/Qwen3-0.6B\", # local path or HF name\n", + " \"lora_r\": 8, \"lora_alpha\": 16,\n", + " \"example_max_seq_len\": 2048,\n", + " \"example_batch_size\": 8,\n", + " \"example_learning_rate\": 2e-4,\n", + "}\n", + "\n", + "# Example 2: Qwen 2.5 7B Instruct - the canonical QLoRA target\n", + "qwen_7b_example = {\n", + " \"model_name\": \"Qwen 2.5 7B Instruct\",\n", + " \"model_path\": \"Qwen/Qwen2.5-7B-Instruct\",\n", + " \"lora_r\": 16, \"lora_alpha\": 32,\n", + " \"example_max_seq_len\": 4096,\n", + " \"example_batch_size\": 16,\n", + " \"example_learning_rate\": 2e-4,\n", + "}\n", + "\n", + "# Example 3: Llama 3.1 8B Instruct\n", + "llama_8b_example = {\n", + " \"model_name\": \"Llama 3.1 8B Instruct\",\n", + " \"model_path\": \"meta-llama/Meta-Llama-3.1-8B-Instruct\",\n", + " \"lora_r\": 16, \"lora_alpha\": 32,\n", + " \"example_max_seq_len\": 4096,\n", + " \"example_batch_size\": 16,\n", + " \"example_learning_rate\": 1e-4,\n", + "}\n", + "\n", + "# =============================================================================\n", + "# SELECT YOUR CONFIGURATION\n", + "# =============================================================================\n", + "selected_example = qwen_small_example\n", + "print(f\"Selected model: {selected_example['model_name']} ({selected_example['model_path']})\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## QLoRA & quantization parameters\n", + "\n", + "`lora_sft(...)` accepts the LoRA hyper-parameters plus the bitsandbytes quantization\n", + "knobs that turn LoRA into QLoRA:\n", + "\n", + "- `load_in_4bit=True` \u2014 load the frozen base model in 4-bit (this is what makes it QLoRA).\n", + "- `bnb_4bit_quant_type=\"nf4\"` \u2014 NormalFloat-4 quantization (recommended over `\"fp4\"`).\n", + "- `bnb_4bit_compute_dtype=\"bfloat16\"` \u2014 de-quantize to bf16 for the matmuls.\n", + "- `bnb_4bit_use_double_quant=True` \u2014 nested quantization of the quant constants (extra memory saving).\n", + "- `lora_r` / `lora_alpha` / `lora_dropout` / `target_modules` \u2014 the LoRA adapter shape." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# =============================================================================\n", + "# COMPLETE QLoRA PARAMETER CONFIGURATION\n", + "# =============================================================================\n", + "from datetime import datetime\n", + "\n", + "experiment_name = \"qlora_comprehensive_example\"\n", + "timestamp = datetime.now().strftime(\"%Y%m%d_%H%M%S\")\n", + "full_experiment_name = f\"{experiment_name}_{timestamp}\"\n", + "\n", + "# ---- Required ----\n", + "model_path = selected_example[\"model_path\"]\n", + "data_path = \"./test_qlora_data.jsonl\"\n", + "ckpt_output_dir = f\"checkpoints/{full_experiment_name}\"\n", + "\n", + "# ---- LoRA adapter shape ----\n", + "lora_r = selected_example[\"lora_r\"] # rank of the low-rank update\n", + "lora_alpha = selected_example[\"lora_alpha\"] # scaling (commonly 2x rank)\n", + "lora_dropout = 0.05\n", + "target_modules = [\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\"] # attention projections; \"all-linear\" also works\n", + "\n", + "# ---- 4-bit quantization (this is what makes it QLoRA) ----\n", + "load_in_4bit = True\n", + "bnb_4bit_quant_type = \"nf4\"\n", + "bnb_4bit_compute_dtype = \"bfloat16\"\n", + "bnb_4bit_use_double_quant = True\n", + "\n", + "# ---- Core training ----\n", + "num_epochs = 3\n", + "effective_batch_size = selected_example[\"example_batch_size\"]\n", + "learning_rate = selected_example[\"example_learning_rate\"] # LoRA tolerates higher LRs than full SFT\n", + "max_seq_len = selected_example[\"example_max_seq_len\"]\n", + "warmup_steps = 10\n", + "\n", + "print(\"QLoRA configuration:\")\n", + "print(f\" 4-bit: load_in_4bit={load_in_4bit}, quant_type={bnb_4bit_quant_type}, \"\n", + " f\"compute_dtype={bnb_4bit_compute_dtype}, double_quant={bnb_4bit_use_double_quant}\")\n", + "print(f\" LoRA: r={lora_r}, alpha={lora_alpha}, dropout={lora_dropout}, targets={target_modules}\")\n", + "print(f\" Train: epochs={num_epochs}, ebs={effective_batch_size}, lr={learning_rate}, max_seq_len={max_seq_len}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Distributed configuration\n", + "\n", + "QLoRA is single-GPU friendly \u2014 that is its main attraction \u2014 but `lora_sft` accepts the\n", + "same distributed topology knobs as `sft`. Keep `nproc_per_node=1` for a single GPU." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Single-GPU is the common QLoRA case.\n", + "nproc_per_node = 1\n", + "nnodes = 1\n", + "node_rank = 0\n", + "rdzv_id = 100\n", + "rdzv_endpoint = \"127.0.0.1:29500\"\n", + "print(f\"distributed: nproc_per_node={nproc_per_node}, nnodes={nnodes}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Execute training\n", + "\n", + "The final cell calls `training_hub.lora_sft(...)`. With `load_in_4bit=True` this performs\n", + "QLoRA: the base model is held in 4-bit NF4 and only the LoRA adapters are trained." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from training_hub import lora_sft\n", + "\n", + "result = lora_sft(\n", + " # --- required ---\n", + " model_path=model_path,\n", + " data_path=data_path,\n", + " ckpt_output_dir=ckpt_output_dir,\n", + " # --- LoRA adapter ---\n", + " lora_r=lora_r,\n", + " lora_alpha=lora_alpha,\n", + " lora_dropout=lora_dropout,\n", + " target_modules=target_modules,\n", + " # --- 4-bit QLoRA quantization ---\n", + " load_in_4bit=load_in_4bit,\n", + " bnb_4bit_quant_type=bnb_4bit_quant_type,\n", + " bnb_4bit_compute_dtype=bnb_4bit_compute_dtype,\n", + " bnb_4bit_use_double_quant=bnb_4bit_use_double_quant,\n", + " # --- core training ---\n", + " num_epochs=num_epochs,\n", + " effective_batch_size=effective_batch_size,\n", + " learning_rate=learning_rate,\n", + " max_seq_len=max_seq_len,\n", + " warmup_steps=warmup_steps,\n", + " # --- distributed ---\n", + " nproc_per_node=nproc_per_node,\n", + " nnodes=nnodes,\n", + " node_rank=node_rank,\n", + " rdzv_id=rdzv_id,\n", + " rdzv_endpoint=rdzv_endpoint,\n", + ")\n", + "print(f\"QLoRA training finished: {result!r}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Post-training analysis\n", + "\n", + "QLoRA writes a **LoRA adapter** (a few MB) rather than a full checkpoint. To serve the\n", + "model you either (a) load the base model + adapter together, or (b) merge the adapter into\n", + "the base weights once and export a standalone checkpoint." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "\n", + "print(f\"Checkpoint dir: {ckpt_output_dir}\")\n", + "if os.path.isdir(ckpt_output_dir):\n", + " for root, _dirs, files in os.walk(ckpt_output_dir):\n", + " for fn in files:\n", + " if fn.endswith((\".safetensors\", \".bin\", \".json\")):\n", + " print(\" \", os.path.relpath(os.path.join(root, fn), ckpt_output_dir))\n", + "\n", + "# Merge the adapter into the base model for standalone inference:\n", + "# from peft import AutoPeftModelForCausalLM\n", + "# m = AutoPeftModelForCausalLM.from_pretrained(\"\")\n", + "# m = m.merge_and_unload()\n", + "# m.save_pretrained(\"merged-model\")\n", + "print(\"\\nTo serve: load base + adapter with peft, or merge_and_unload() for a standalone model.\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Parameter reference summary\n", + "\n", + "### QLoRA-specific parameters\n", + "\n", + "| Parameter | Description | Typical |\n", + "|---|---|---|\n", + "| `load_in_4bit` | Enable 4-bit base \u2014 turns LoRA into QLoRA | `True` |\n", + "| `bnb_4bit_quant_type` | 4-bit data type | `\"nf4\"` |\n", + "| `bnb_4bit_compute_dtype` | Matmul compute dtype | `\"bfloat16\"` |\n", + "| `bnb_4bit_use_double_quant` | Nested quantization | `True` |\n", + "| `lora_r` | LoRA rank | 8-64 |\n", + "| `lora_alpha` | LoRA scaling | 2x `lora_r` |\n", + "| `lora_dropout` | Adapter dropout | 0.0-0.1 |\n", + "| `target_modules` | Layers to adapt | attention proj / `\"all-linear\"` |\n", + "| `load_in_8bit` | 8-bit base (LoRA-8bit, not QLoRA) | `False` |\n", + "\n", + "### Common training parameters\n", + "\n", + "| Parameter | Description |\n", + "|---|---|\n", + "| `model_path`, `data_path`, `ckpt_output_dir` | Required I/O |\n", + "| `num_epochs`, `effective_batch_size`, `learning_rate`, `max_seq_len` | Core training |\n", + "| `warmup_steps`, `lr_scheduler` | LR schedule |\n", + "| `nproc_per_node`, `nnodes`, `node_rank`, `rdzv_id`, `rdzv_endpoint` | Distributed topology |\n", + "\n", + "> **Hardware note:** 4-bit QLoRA via bitsandbytes requires an NVIDIA GPU of compute\n", + "> capability **sm_75 or newer** (Turing/Ampere/Hopper). Older cards (e.g. P100, sm_60)\n", + "> are not supported by the bitsandbytes 4-bit kernels.\n", + ">\n", + "> **NPU note:** On Huawei Ascend NPU, bitsandbytes 4-bit is not available. Use LoRA\n", + "> (without 4-bit) on the `llamafactory0.9-cann8.5-arm64` runtime\n", + "> (`finetuning_type: lora`) as the parameter-efficient path \u2014 see\n", + "> [Fine-tune and Pretrain on Ascend NPU](../fine-tune-and-pretrain-llms-on-ascend-npu.mdx)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.11" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/en/training_guides/training-hub-fine-tuning.mdx b/docs/en/training_guides/training-hub-fine-tuning.mdx index 31f42e4..42f5fe5 100644 --- a/docs/en/training_guides/training-hub-fine-tuning.mdx +++ b/docs/en/training_guides/training-hub-fine-tuning.mdx @@ -4,18 +4,33 @@ weight: 50 # Fine-tuning LLMs with Training Hub -`training_hub` is a Python library that wraps Supervised Fine-Tuning (SFT) and Orthogonal Subspace Fine-Tuning (OSFT) behind a single function call (`sft(...)`, `osft(...)`) that handles single-GPU, multi-GPU, and multi-node training uniformly. +`training_hub` is a Python library that wraps several LLM post-training algorithms — Supervised Fine-Tuning (SFT), Orthogonal Subspace Fine-Tuning (OSFT), LoRA / QLoRA, and continued pre-training (CPT) — behind single function calls (`sft(...)`, `osft(...)`, `lora_sft(...)`) that handle single-GPU, multi-GPU, and multi-node training uniformly. - **Automatic memory management** — `max_tokens_per_gpu` caps GPU memory and auto-computes micro-batch size and gradient accumulation to hit your target `effective_batch_size`. - **OSFT** implements [Nayak et al., 2025 (arXiv:2504.07097)](https://arxiv.org/abs/2504.07097) — restricting weight updates to orthogonal subspaces prevents catastrophic forgetting without replay data. +- **QLoRA** loads the frozen base model in 4-bit and trains only LoRA adapters, fitting large models on a single small GPU slice ([Dettmers et al., 2023, arXiv:2305.14314](https://arxiv.org/abs/2305.14314)). +- **Continued pre-training (CPT)** runs next-token prediction over a raw-text corpus to inject domain knowledge before instruction tuning. - Built-in checkpointing, experiment tracking, and Liger kernel support. -| Aspect | SFT | OSFT | -|--------|-----|------| -| Use case | Initial instruction tuning | Continual domain adaptation of tuned models | -| Forgetting mitigation | Mix/replay data | Algorithmic (orthogonal subspaces) | -| Key parameter | Standard hyperparameters | `unfreeze_rank_ratio` (0.0–1.0) | -| Backend | instructlab-training | mini-trainer | +:::note Design context +`training_hub` is the upstream library RHOAI / Open Data Hub uses to expose post-training +algorithms behind a single API, and Alauda AI runs the same code on Kubeflow Trainer v2 +(`TrainJob` / `ClusterTrainingRuntime`). The split between the *algorithm* (this library) +and the *distributed runtime* (Kubeflow Trainer) follows the Open Data Hub +[architecture decision records](https://github.com/opendatahub-io/architecture-decision-records/tree/main/documentation) +(see the `distributed-workload` and `workbenches` component docs) — so the recipes here +map cleanly onto the same `sft` / `osft` / `lora_sft` entrypoints whether you run them in a +workbench notebook or as a cluster `TrainJob`. +::: + +| Aspect | SFT | OSFT | QLoRA | CPT | +|--------|-----|------|-------|-----| +| Use case | Initial instruction tuning | Continual domain adaptation of tuned models | Memory-efficient adaptation | Domain knowledge injection | +| Data | Chat JSONL | Chat JSONL | Chat JSONL | Raw text | +| Trainable weights | All (bf16) | Orthogonal subspace | LoRA adapters (base in 4-bit) | All (bf16) | +| Key parameter | Standard hyperparameters | `unfreeze_rank_ratio` (0.0–1.0) | `load_in_4bit` + `lora_r` | `is_pretraining` + `block_size` | +| Entrypoint | `sft(...)` | `osft(...)` | `lora_sft(...)` | `sft(..., is_pretraining=True)` | +| Backend | instructlab-training | mini-trainer | unsloth / peft + bitsandbytes | instructlab-training | ## Requirements @@ -47,6 +62,8 @@ Download into your workbench and execute cell-by-cell: |---|---|---| | SFT comprehensive tutorial | SFT | [`sft-comprehensive-tutorial.ipynb`](https://github.com/alauda/aml-docs/tree/master/docs/en/training_guides/sft-comprehensive-tutorial.ipynb) | | OSFT comprehensive tutorial | OSFT | [`osft-comprehensive-tutorial.ipynb`](https://github.com/alauda/aml-docs/tree/master/docs/en/training_guides/osft-comprehensive-tutorial.ipynb) | +| QLoRA comprehensive tutorial | QLoRA | [`qlora-comprehensive-tutorial.ipynb`](https://github.com/alauda/aml-docs/tree/master/docs/en/training_guides/qlora-comprehensive-tutorial.ipynb) | +| CPT comprehensive tutorial | CPT | [`cpt-comprehensive-tutorial.ipynb`](https://github.com/alauda/aml-docs/tree/master/docs/en/training_guides/cpt-comprehensive-tutorial.ipynb) | Install and configure: @@ -131,6 +148,118 @@ OSFT-only: | `unmask_messages` | No | If `True`, train on all non-system content | | `target_patterns` | No | Substring patterns to restrict OSFT to specific layers | +## QLoRA (4-bit LoRA) \{#qlora} + +QLoRA freezes the base model in **4-bit NormalFloat (NF4)** precision and trains only small +**LoRA adapter** matrices on top. A 7B model that needs ~60 GiB for full SFT then fits on a +single 16–24 GiB GPU (or HAMI vGPU slice). Use it whenever you are GPU-memory bound; the cost +is a small quantization quality gap and slightly slower steps. + +`training_hub` exposes QLoRA through `lora_sft(...)` — LoRA plus the bitsandbytes 4-bit knobs: + +```python +from training_hub import lora_sft + +result = lora_sft( + model_path="Qwen/Qwen2.5-7B-Instruct", # HF name or local path + data_path="/path/to/training_data.jsonl", # same chat JSONL as SFT + ckpt_output_dir="/path/to/checkpoints/qlora_run", + # LoRA adapter + lora_r=16, lora_alpha=32, lora_dropout=0.05, + target_modules=["q_proj", "k_proj", "v_proj", "o_proj"], # or "all-linear" + # 4-bit quantization — this is what makes it QLoRA + load_in_4bit=True, + bnb_4bit_quant_type="nf4", + bnb_4bit_compute_dtype="bfloat16", + bnb_4bit_use_double_quant=True, + # core training (LoRA tolerates higher LRs than full SFT) + num_epochs=3, effective_batch_size=16, learning_rate=2e-4, max_seq_len=4096, + nproc_per_node=1, +) +``` + +QLoRA-specific parameters: + +| Parameter | Required | Description | +|---|---|---| +| `load_in_4bit` | Yes (for QLoRA) | Load the frozen base model in 4-bit. Without it `lora_sft` is plain (16-bit) LoRA. | +| `bnb_4bit_quant_type` | No | `"nf4"` (recommended) or `"fp4"` | +| `bnb_4bit_compute_dtype` | No | De-quantization compute dtype, e.g. `"bfloat16"` | +| `bnb_4bit_use_double_quant` | No | Nested quantization of the quant constants (extra memory saving) | +| `lora_r`, `lora_alpha`, `lora_dropout` | Yes | LoRA adapter rank / scaling / dropout | +| `target_modules` | No | Modules to adapt (attention projections, or `"all-linear"`) | +| `load_in_8bit` | No | 8-bit base (LoRA-8bit, not QLoRA) | + +The output is a **LoRA adapter**, not a full checkpoint. To serve, load base + adapter with +`peft`, or merge once with `merge_and_unload()` and export a standalone model. + +:::note +4-bit QLoRA via bitsandbytes needs an NVIDIA GPU of compute capability **sm_75 or newer** +(Turing / Ampere / Hopper). The `traininghub0.1-cu126-amd64` runtime already bundles +`trl`, `peft`, and `bitsandbytes`. The default `lora_sft` backend is `unsloth`; for full +control you can also drive `peft` + `bitsandbytes` directly on the same runtime image. + +**NPU:** bitsandbytes 4-bit is not available on Huawei Ascend. Use LoRA (without 4-bit) on +the `llamafactory0.9-cann8.5-arm64` runtime (`finetuning_type: lora`) as the +parameter-efficient path — see [Fine-tune and Pretrain on Ascend NPU](./fine-tune-and-pretrain-llms-on-ascend-npu.mdx). +::: + +## Continued pre-training (CPT) \{#cpt} + +Continued pre-training (CPT) keeps the original **next-token prediction** objective but runs +it over a **raw-text corpus** from your domain (medical, legal, code, a new language). It +injects *knowledge and vocabulary* — unlike SFT/OSFT, which teach *behavior* from chat data. +A common pipeline is **CPT → SFT → alignment**. + +`training_hub` runs CPT through the same `sft(...)` entrypoint with `is_pretraining=True`: +the loss covers *all* tokens (no assistant-only masking) and documents are packed into +fixed-size `block_size` windows. + +```python +from training_hub import sft + +result = sft( + model_path="Qwen/Qwen2.5-7B", # a BASE checkpoint + data_path="/path/to/corpus.jsonl", # raw text, one document per line + ckpt_output_dir="/path/to/checkpoints/cpt_run", + # continued pre-training + is_pretraining=True, + block_size=4096, # packed context-window length + document_column_name="text", # JSONL field holding the raw text + # core training — keep the LR low to limit forgetting + num_epochs=1, effective_batch_size=128, learning_rate=5e-6, + max_seq_len=4096, max_tokens_per_gpu=20000, + nproc_per_node=8, +) +``` + +CPT data is raw text, **not** chat turns — one document per line under `document_column_name`: + +```json +{"text": "A paragraph or document of raw domain text ..."} +{"text": "Another document ..."} +``` + +CPT-specific parameters: + +| Parameter | Required | Description | +|---|---|---| +| `is_pretraining` | Yes | Switch to raw-text pre-training (no chat masking) | +| `block_size` | No | Packed context-window length (e.g. 1024–4096) | +| `document_column_name` | No | JSONL field holding the raw text (default `"text"`) | + +CPT updates **all** weights and writes a full checkpoint. It can cause catastrophic +forgetting of general ability — mitigate with a low learning rate, by mixing in some +general-domain text, and by following up with an SFT/OSFT pass (point `model_path` at the CPT +checkpoint). If forgetting is the primary concern, prefer **OSFT**. + +:::note +**NPU:** Full-parameter continued pre-training also runs on Huawei Ascend via the +`MindSpeed-LLM` runtime (`pretrain_gpt.py`). See +[Fine-tune and Pretrain on Ascend NPU](./fine-tune-and-pretrain-llms-on-ascend-npu.mdx) and +the `qwen25_pretrain_verify.ipynb` recipe. +::: + ## Multi-node Run the notebook (or script) on every node with the same `rdzv_id` / `rdzv_endpoint` and varying `node_rank`: diff --git a/e2e/cases/c13_traininghub_qlora.sh b/e2e/cases/c13_traininghub_qlora.sh new file mode 100755 index 0000000..a651ba5 --- /dev/null +++ b/e2e/cases/c13_traininghub_qlora.sh @@ -0,0 +1,306 @@ +#!/usr/bin/env bash +# C13 — exercises QLoRA (4-bit NF4 LoRA) on the published +# traininghub0.1-cu126-amd64 runtime image (the runtime that bundles +# trl + peft + bitsandbytes). Drives training_hub.lora_sft(load_in_4bit=True,...) +# as the canonical Training Hub QLoRA path, with a trl+peft+bitsandbytes fallback. +# A tiny synthetic Qwen2 checkpoint + synthetic chat JSONL are generated inside the +# Pod so the case has no external/model-download dependency (matches c3/c4). +# +# IMAGE: defaults to the cluster-pullable build-harbor mirror — docker.io is +# EGRESS-BLOCKED on the GPU cluster nodes, so the dockerhub tag +# (docker.io/alaudadockerhub/traininghub0.1-cu126-amd64:v0.1.0) ImagePullBackOffs +# there. build-harbor needs the `harbor-mlops-regcred` pull secret in the run +# namespace; this case auto-creates it from $ACP_HARBOR_USER/$ACP_HARBOR_PASS when +# E2E_IMAGE_PULL_SECRET is unset and those creds are present (see ensure_pull_secret). +# +# SKIP (rc=77) conditions, both captured from real cluster output: +# * the requested HAMI vGPU slice cannot be scheduled (e.g. the only Ampere+ +# GPU is fully reserved) -> CardInsufficientMemory / Unschedulable; +# * the GPU that *is* available is older than sm_75 (e.g. P100 sm_60), which the +# bitsandbytes 4-bit kernels do not support -> in-pod arch guard exits 77. +set -euo pipefail +HERE="$(cd "$(dirname "$0")" && pwd)" +source "${HERE}/../lib.sh" + +require_env GPU_NAMESPACE "namespace for GPU e2e resources" +NS="${GPU_NAMESPACE}" +JOB_NAME="c13-traininghub-qlora-$(printf '%05x' $$)" +# Cluster-pullable by default (docker.io is egress-blocked on the GPU nodes). +# Faster intra-cluster mirror: 152-231-registry.alauda.cn:60070/mlops/traininghub0.1-cu126-amd64:v0.1.0-build.20260609030710 +IMAGE="${C13_IMAGE:-build-harbor.alauda.cn/mlops/traininghub0.1-cu126-amd64:v0.1.0-build.20260609030710}" +# Ensure (create-if-missing) the harbor pull secret, then default to it. +IMAGE_PULL_SECRET="${C13_IMAGE_PULL_SECRET:-${E2E_IMAGE_PULL_SECRET:-$(ensure_pull_secret "${NS}")}}" + +cleanup() { + gpu_kc -n "${NS}" delete job "${JOB_NAME}" --ignore-not-found --wait=false || true +} +trap cleanup EXIT + +log "C13: submitting Job ${JOB_NAME} (image=${IMAGE})" + +cat < SKIP(77). + python - <<'PY' + import sys, torch + if not torch.cuda.is_available(): + print("E2E-SKIP: no CUDA device visible", file=sys.stderr); sys.exit(77) + cc = torch.cuda.get_device_capability(0); sm = cc[0]*10 + cc[1] + name = torch.cuda.get_device_name(0) + print(f"GPU={name} capability=sm_{sm}") + if sm < 75: + print(f"E2E-SKIP: bitsandbytes 4-bit QLoRA requires sm_75+, got {name} sm_{sm}", file=sys.stderr) + sys.exit(77) + PY + + # 1) Build a tiny synthetic Qwen2-style causal LM + tokenizer (offline). + python - <<'PY' + import os, json, torch + from transformers import GPT2TokenizerFast, Qwen2Config, Qwen2ForCausalLM + from tokenizers import ByteLevelBPETokenizer + + MODEL_DIR = "/workspace/tiny-qwen2" + os.makedirs(MODEL_DIR, exist_ok=True) + torch.manual_seed(0) + bpe = ByteLevelBPETokenizer() + bpe.train_from_iterator( + ["hello world", "the quick brown fox", "alauda ai e2e"], vocab_size=512, + min_frequency=1, special_tokens=["", "", "", ""], + ) + bpe.save_model(MODEL_DIR) + tok = GPT2TokenizerFast( + vocab_file=f"{MODEL_DIR}/vocab.json", + merges_file=f"{MODEL_DIR}/merges.txt", + unk_token="", bos_token="", eos_token="", pad_token="", + ) + tok.chat_template = "{% for m in messages %}{{ m.role }}: {{ m.content }}\n{% endfor %}" + tok.save_pretrained(MODEL_DIR) + cfg = Qwen2Config( + vocab_size=tok.vocab_size + 4, + hidden_size=64, num_hidden_layers=2, num_attention_heads=4, + num_key_value_heads=4, intermediate_size=128, max_position_embeddings=256, + rope_theta=10000.0, tie_word_embeddings=True, + ) + Qwen2ForCausalLM(cfg).save_pretrained(MODEL_DIR) + + data_path = "/workspace/test_qlora_data.jsonl" + with open(data_path, "w") as f: + for _ in range(16): + json.dump({"messages": [ + {"role": "system", "content": "You are a helpful assistant."}, + {"role": "user", "content": "Hello, how are you?"}, + {"role": "assistant", "content": "I am well - how can I help?"}, + ]}, f); f.write("\n") + print(f"prepared {MODEL_DIR} and {data_path}") + PY + + # 2) QLoRA: training_hub.lora_sft(load_in_4bit) is the canonical path; + # fall back to trl+peft+bitsandbytes (bundled in this runtime) if the + # unsloth backend can't load the offline synthetic model. + python - <<'PY' + import os, time, glob + MODEL_DIR = "/workspace/tiny-qwen2" + DATA = "/workspace/test_qlora_data.jsonl" + CKPT = "/workspace/ckpt" + + def has_adapter(root): + hits = glob.glob(os.path.join(root, "**", "adapter_model*"), recursive=True) + hits += glob.glob(os.path.join(root, "**", "*.safetensors"), recursive=True) + return hits + + ok = False + try: + from training_hub import lora_sft + t0 = time.time() + result = lora_sft( + model_path=MODEL_DIR, data_path=DATA, ckpt_output_dir=CKPT, + lora_r=8, lora_alpha=16, lora_dropout=0.05, + target_modules=["q_proj", "k_proj", "v_proj", "o_proj"], + load_in_4bit=True, bnb_4bit_quant_type="nf4", + bnb_4bit_compute_dtype="bfloat16", bnb_4bit_use_double_quant=True, + num_epochs=1, effective_batch_size=2, learning_rate=2e-4, + max_seq_len=128, warmup_steps=0, + nproc_per_node=1, nnodes=1, node_rank=0, + rdzv_id=13, rdzv_endpoint="127.0.0.1:29513", + ) + print(f"training_hub.lora_sft finished in {time.time()-t0:.1f}s: {result!r}") + ok = bool(has_adapter(CKPT)) + except Exception as e: + print(f"training_hub.lora_sft path unavailable ({type(e).__name__}: {e}); " + f"falling back to trl+peft+bitsandbytes QLoRA") + + if not ok: + import torch + from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig + from peft import LoraConfig, prepare_model_for_kbit_training + from trl import SFTConfig, SFTTrainer + from datasets import load_dataset + + bnb = BitsAndBytesConfig( + load_in_4bit=True, bnb_4bit_quant_type="nf4", + bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True, + ) + tok = AutoTokenizer.from_pretrained(MODEL_DIR) + model = AutoModelForCausalLM.from_pretrained( + MODEL_DIR, quantization_config=bnb, device_map={"": 0}, + attn_implementation="eager", + ) + model = prepare_model_for_kbit_training(model) + peft_cfg = LoraConfig( + r=8, lora_alpha=16, lora_dropout=0.05, bias="none", + target_modules=["q_proj", "k_proj", "v_proj", "o_proj"], + task_type="CAUSAL_LM", + ) + ds = load_dataset("json", data_files=DATA, split="train") + ds = ds.map(lambda ex: {"text": tok.apply_chat_template(ex["messages"], tokenize=False)}) + + # trl 0.17: max_seq_length lives on SFTConfig; tolerate field renames. + cfg_kw = dict(output_dir=CKPT, num_train_epochs=1, per_device_train_batch_size=2, + max_steps=5, logging_steps=1, learning_rate=2e-4, + report_to=[], bf16=True, dataset_text_field="text") + try: + cfg = SFTConfig(max_seq_length=128, **cfg_kw) + except TypeError: + cfg = SFTConfig(max_length=128, **cfg_kw) + try: + trainer = SFTTrainer(model=model, args=cfg, train_dataset=ds, + peft_config=peft_cfg, processing_class=tok) + except TypeError: + trainer = SFTTrainer(model=model, args=cfg, train_dataset=ds, + peft_config=peft_cfg, tokenizer=tok) + t0 = time.time() + trainer.train() + trainer.save_model(CKPT) + print(f"trl QLoRA finished in {time.time()-t0:.1f}s") + ok = bool(has_adapter(CKPT)) + + adapters = has_adapter(CKPT) + assert adapters, f"no QLoRA adapter/checkpoint under {CKPT}" + print(f"QLoRA artifacts: {[os.path.relpath(a, CKPT) for a in adapters][:5]}") + PY +YAML + +log "C13: waiting for pod to appear..." +deadline=$((SECONDS + 120)) +POD="" +while [ "${SECONDS}" -lt "${deadline}" ]; do + POD="$(gpu_kc -n "${NS}" get pod -l "job-name=${JOB_NAME}" -o jsonpath='{.items[0].metadata.name}' 2>/dev/null || true)" + [ -n "${POD}" ] && break + sleep 5 +done +log "C13: pod=${POD}" + +# Scheduling SKIP: if the pod never leaves Pending (no schedulable GPU slice), +# capture the real scheduler event and SKIP rather than fail. +sched_deadline=$((SECONDS + 300)) +phase="" +while [ "${SECONDS}" -lt "${sched_deadline}" ]; do + phase="$(gpu_kc -n "${NS}" get pod "${POD}" -o jsonpath='{.status.phase}' 2>/dev/null || true)" + case "${phase}" in Running|Succeeded|Failed) break ;; esac + sleep 5 +done +if [ "${phase}" = "Pending" ] || [ -z "${phase}" ]; then + EVT="$(gpu_kc -n "${NS}" get event --field-selector "involvedObject.name=${POD}" \ + -o jsonpath='{range .items[*]}{.reason}: {.message}{"\n"}{end}' 2>/dev/null | tail -3)" + if echo "${EVT}" | grep -qiE 'CardInsufficientMemory|Insufficient|Unschedulable|FailedScheduling|FilteringFailed|untolerated|no available node'; then + log "C13: SKIP — pod ${POD} cannot be scheduled onto a GPU slice:" + echo "${EVT}" | sed 's/^/ /' + exit "${E2E_SKIP_RC}" + fi + log "C13: SKIP — pod ${POD} still ${phase:-} after scheduling deadline:" + echo "${EVT}" | sed 's/^/ /' + exit "${E2E_SKIP_RC}" +fi + +# Stream logs and wait for terminal job state. +if [ -n "${POD}" ]; then + gpu_kc -n "${NS}" logs -f "${POD}" 2>&1 & + LOGS_PID=$! +fi +deadline=$((SECONDS + 2400)) +status="" +while [ "${SECONDS}" -lt "${deadline}" ]; do + status="$(gpu_kc -n "${NS}" get job "${JOB_NAME}" -o jsonpath='{.status.conditions[?(@.status=="True")].type}' 2>/dev/null || true)" + case "${status}" in *Complete*) break ;; *Failed*) break ;; esac + sleep 15 +done +[ -n "${LOGS_PID:-}" ] && wait "${LOGS_PID}" 2>/dev/null || true +log "C13: job status=${status}" + +# Runtime SKIP: the in-pod arch guard exits 77 on unsupported GPUs (sm < 75). +EXIT_CODE="$(gpu_kc -n "${NS}" get pod "${POD}" -o jsonpath='{.status.containerStatuses[0].state.terminated.exitCode}' 2>/dev/null || true)" +if [ "${EXIT_CODE}" = "77" ]; then + log "C13: SKIP — in-pod guard signalled unsupported GPU for 4-bit QLoRA:" + gpu_kc -n "${NS}" logs "${POD}" --tail=20 2>&1 | grep -i 'E2E-SKIP' | sed 's/^/ /' || true + exit "${E2E_SKIP_RC}" +fi + +if [[ "${status}" != *Complete* ]]; then + log "C13: ==== pod final state ====" + gpu_kc -n "${NS}" get pod -l "job-name=${JOB_NAME}" -o wide 2>&1 | tail -5 || true + log "C13: ==== container logs ====" + gpu_kc -n "${NS}" logs -l "job-name=${JOB_NAME}" --tail=200 2>&1 || true + log "C13: ==== pod describe ====" + gpu_kc -n "${NS}" describe pod -l "job-name=${JOB_NAME}" 2>&1 | tail -30 || true +fi +[[ "${status}" == *Complete* ]] diff --git a/e2e/cases/c14_traininghub_cpt.sh b/e2e/cases/c14_traininghub_cpt.sh new file mode 100755 index 0000000..a4340fa --- /dev/null +++ b/e2e/cases/c14_traininghub_cpt.sh @@ -0,0 +1,228 @@ +#!/usr/bin/env bash +# C14 — exercises continued pre-training (CPT) on the published +# traininghub0.1-cu126-amd64 runtime image. Drives +# training_hub.sft(is_pretraining=True, block_size=..., document_column_name="text") +# over a tiny synthetic RAW-TEXT corpus + tiny synthetic Qwen2 checkpoint, both +# generated inside the Pod (no external/model/corpus download — matches c3/c4). +# CPT is full-parameter and uses torch SDPA, so (unlike c13 QLoRA) it has no +# sm_75 requirement; flash_attn is force-disabled for older-GPU compatibility. +# +# IMAGE: defaults to the cluster-pullable build-harbor mirror — docker.io is +# EGRESS-BLOCKED on the GPU cluster nodes (the dockerhub tag ImagePullBackOffs). +# build-harbor needs the `harbor-mlops-regcred` pull secret in the run namespace; +# this case auto-creates it from $ACP_HARBOR_USER/$ACP_HARBOR_PASS when +# E2E_IMAGE_PULL_SECRET is unset and those creds are present (see ensure_pull_secret). +# +# SKIP (rc=77): the requested HAMI vGPU slice cannot be scheduled (the captured +# scheduler event, e.g. CardInsufficientMemory / Unschedulable, is printed). +set -euo pipefail +HERE="$(cd "$(dirname "$0")" && pwd)" +source "${HERE}/../lib.sh" + +require_env GPU_NAMESPACE "namespace for GPU e2e resources" +NS="${GPU_NAMESPACE}" +JOB_NAME="c14-traininghub-cpt-$(printf '%05x' $$)" +# Cluster-pullable by default (docker.io is egress-blocked on the GPU nodes). +# Faster intra-cluster mirror: 152-231-registry.alauda.cn:60070/mlops/traininghub0.1-cu126-amd64:v0.1.0-build.20260609030710 +IMAGE="${C14_IMAGE:-build-harbor.alauda.cn/mlops/traininghub0.1-cu126-amd64:v0.1.0-build.20260609030710}" +# Ensure (create-if-missing) the harbor pull secret, then default to it. +IMAGE_PULL_SECRET="${C14_IMAGE_PULL_SECRET:-${E2E_IMAGE_PULL_SECRET:-$(ensure_pull_secret "${NS}")}}" + +cleanup() { + gpu_kc -n "${NS}" delete job "${JOB_NAME}" --ignore-not-found --wait=false || true +} +trap cleanup EXIT + +log "C14: submitting Job ${JOB_NAME} (image=${IMAGE})" + +cat <", "", "", ""], + ) + bpe.save_model(MODEL_DIR) + tok = GPT2TokenizerFast( + vocab_file=f"{MODEL_DIR}/vocab.json", + merges_file=f"{MODEL_DIR}/merges.txt", + unk_token="", bos_token="", eos_token="", pad_token="", + ) + tok.chat_template = "{% for m in messages %}{{ m.role }}: {{ m.content }}\n{% endfor %}" + tok.save_pretrained(MODEL_DIR) + cfg = Qwen2Config( + vocab_size=tok.vocab_size + 4, + hidden_size=64, num_hidden_layers=2, num_attention_heads=4, + num_key_value_heads=4, intermediate_size=128, max_position_embeddings=256, + rope_theta=10000.0, tie_word_embeddings=True, + ) + Qwen2ForCausalLM(cfg).save_pretrained(MODEL_DIR) + + # 2) Synthetic RAW-TEXT corpus: one document per line under "text". + data_path = "/workspace/test_cpt_data.jsonl" + docs = [ + "Alauda AI is an MLOps platform that runs fine-tuning and inference on Kubernetes.", + "Continued pre-training adapts a base language model to a new domain using unlabeled text.", + "The training hub library wraps SFT, OSFT, LoRA, QLoRA and continued pre-training.", + "Kubeflow Trainer v2 submits distributed TrainJobs onto GPU or NPU nodes.", + ] * 8 + with open(data_path, "w") as f: + for d in docs: + json.dump({"text": d}, f); f.write("\n") + print(f"prepared base model {MODEL_DIR} and raw-text corpus {data_path}") + PY + + # 3) Continued pre-training via training_hub.sft(is_pretraining=True). + python - <<'PY' + import os, time + from training_hub import sft + t0 = time.time() + result = sft( + model_path="/workspace/tiny-qwen2", + data_path="/workspace/test_cpt_data.jsonl", + ckpt_output_dir="/workspace/ckpt", + # --- continued pre-training (CPT) --- + is_pretraining=True, + block_size=128, + document_column_name="text", + # --- core training --- + num_epochs=1, + effective_batch_size=2, + learning_rate=5e-6, + max_seq_len=128, + max_tokens_per_gpu=256, + data_output_dir="/workspace/data_cache", + warmup_steps=0, + checkpoint_at_epoch=True, + accelerate_full_state_at_epoch=False, + nproc_per_node=1, nnodes=1, node_rank=0, + rdzv_id=14, rdzv_endpoint="127.0.0.1:29514", + disable_flash_attn=True, + use_liger=False, + ) + print(f"CPT (sft is_pretraining) finished in {time.time()-t0:.1f}s: {result!r}") + hf_dir = "/workspace/ckpt/hf_format" + assert os.path.isdir(hf_dir), f"no hf_format dir at {hf_dir}" + ckpts = sorted(os.listdir(hf_dir)) + assert ckpts, f"no checkpoints under {hf_dir}" + print(f"checkpoints: {ckpts}") + PY +YAML + +log "C14: waiting for pod to appear..." +deadline=$((SECONDS + 120)) +POD="" +while [ "${SECONDS}" -lt "${deadline}" ]; do + POD="$(gpu_kc -n "${NS}" get pod -l "job-name=${JOB_NAME}" -o jsonpath='{.items[0].metadata.name}' 2>/dev/null || true)" + [ -n "${POD}" ] && break + sleep 5 +done +log "C14: pod=${POD}" + +# Scheduling SKIP: if no schedulable GPU slice, capture the scheduler event and SKIP. +sched_deadline=$((SECONDS + 300)) +phase="" +while [ "${SECONDS}" -lt "${sched_deadline}" ]; do + phase="$(gpu_kc -n "${NS}" get pod "${POD}" -o jsonpath='{.status.phase}' 2>/dev/null || true)" + case "${phase}" in Running|Succeeded|Failed) break ;; esac + sleep 5 +done +if [ "${phase}" = "Pending" ] || [ -z "${phase}" ]; then + EVT="$(gpu_kc -n "${NS}" get event --field-selector "involvedObject.name=${POD}" \ + -o jsonpath='{range .items[*]}{.reason}: {.message}{"\n"}{end}' 2>/dev/null | tail -3)" + log "C14: SKIP — pod ${POD} still ${phase:-} (no schedulable GPU slice):" + echo "${EVT}" | sed 's/^/ /' + exit "${E2E_SKIP_RC}" +fi + +if [ -n "${POD}" ]; then + gpu_kc -n "${NS}" logs -f "${POD}" 2>&1 & + LOGS_PID=$! +fi +deadline=$((SECONDS + 2400)) +status="" +while [ "${SECONDS}" -lt "${deadline}" ]; do + status="$(gpu_kc -n "${NS}" get job "${JOB_NAME}" -o jsonpath='{.status.conditions[?(@.status=="True")].type}' 2>/dev/null || true)" + case "${status}" in *Complete*) break ;; *Failed*) break ;; esac + sleep 15 +done +[ -n "${LOGS_PID:-}" ] && wait "${LOGS_PID}" 2>/dev/null || true +log "C14: job status=${status}" +if [[ "${status}" != *Complete* ]]; then + log "C14: ==== pod final state ====" + gpu_kc -n "${NS}" get pod -l "job-name=${JOB_NAME}" -o wide 2>&1 | tail -5 || true + log "C14: ==== container logs ====" + gpu_kc -n "${NS}" logs -l "job-name=${JOB_NAME}" --tail=200 2>&1 || true + log "C14: ==== pod describe ====" + gpu_kc -n "${NS}" describe pod -l "job-name=${JOB_NAME}" 2>&1 | tail -30 || true +fi +[[ "${status}" == *Complete* ]] diff --git a/e2e/lib.sh b/e2e/lib.sh index 435887a..52fa826 100644 --- a/e2e/lib.sh +++ b/e2e/lib.sh @@ -95,6 +95,42 @@ yaml_image_pull_secrets() { printf '%*s- name: %s\n' "$((indent + 2))" '' "${secret}" } +# Ensure a build-harbor.alauda.cn pull secret exists in the given namespace and +# echo its name (for use as imagePullSecrets). docker.io is egress-blocked on the +# GPU cluster nodes, so the traininghub/llamafactory runtimes must be pulled from +# build-harbor, which is private and needs dockerconfigjson creds. +# +# Resolution order (echoes the secret NAME on stdout; diagnostics go to stderr): +# 1. If a secret named ${HARBOR_PULL_SECRET:-harbor-mlops-regcred} already exists +# in the namespace, reuse it (the orchestrator/cluster usually pre-seeds it). +# 2. Else, if $ACP_HARBOR_USER and $ACP_HARBOR_PASS are set, create it. +# 3. Else, echo nothing (caller falls through to no pull secret) and warn — the +# orchestrator must pre-create it; the Pod will ImagePullBackOff otherwise. +ensure_pull_secret() { + local ns="$1" + local name="${HARBOR_PULL_SECRET:-harbor-mlops-regcred}" + local server="${HARBOR_SERVER:-build-harbor.alauda.cn}" + if gpu_kc -n "${ns}" get secret "${name}" >/dev/null 2>&1; then + log "ensure_pull_secret: reusing existing secret ${name} in ns ${ns}" >&2 + printf '%s' "${name}" + return 0 + fi + if [ -n "${ACP_HARBOR_USER:-}" ] && [ -n "${ACP_HARBOR_PASS:-}" ]; then + if gpu_kc -n "${ns}" create secret docker-registry "${name}" \ + --docker-server="${server}" \ + --docker-username="${ACP_HARBOR_USER}" \ + --docker-password="${ACP_HARBOR_PASS}" >/dev/null 2>&1; then + log "ensure_pull_secret: created ${name} (server=${server}) in ns ${ns}" >&2 + printf '%s' "${name}" + return 0 + fi + log "ensure_pull_secret: failed to create ${name} in ns ${ns}" >&2 + fi + log "ensure_pull_secret: no pull secret available (set E2E_IMAGE_PULL_SECRET or ACP_HARBOR_USER/PASS, or pre-create ${name} for ${server} in ns ${ns})" >&2 + printf '' + return 0 +} + yaml_node_selector() { local indent="$1" key="${2:-}" value="${3:-}" [ -z "${key}" ] && return 0 diff --git a/e2e/run_all.sh b/e2e/run_all.sh index b4097d8..64fe194 100755 --- a/e2e/run_all.sh +++ b/e2e/run_all.sh @@ -29,6 +29,18 @@ CASES=( # C12 needs Kueue installed; it skips with rc=77 if the kueue.x-k8s.io API # group is missing. See preemptible-trainjobs-with-kueue.mdx. "C12:GPU:cases/c12_kueue_preemption.sh" + # C13 — QLoRA (4-bit NF4 LoRA) via training_hub.lora_sft on the traininghub + # runtime. Self-contained synthetic model; SKIPs (rc=77) if no Ampere+ GPU + # slice is schedulable or the available GPU is < sm_75 (bitsandbytes 4-bit). + # Defaults to the cluster-pullable build-harbor image (docker.io is blocked). + "C13:GPU:cases/c13_traininghub_qlora.sh" + # C14 — continued pre-training (CPT) via training_hub.sft(is_pretraining=True). + # Self-contained (synthetic base model + synthetic raw-text corpus, no fetch). + # CPT is full-parameter SDPA — no sm_75 floor, so it runs on any schedulable + # GPU slice (Ampere/Hopper/Pascal). SKIPs (rc=77) when the only Ampere GPU + # (A30) is reserved by the persistent inference workload and no slice frees up; + # the orchestrator controls A30 capacity. Same build-harbor image as C13. + "C14:GPU:cases/c14_traininghub_cpt.sh" ) want=( "$@" )