Lora Weights not Loaded from Checkpoint

[maliozer]

Describe the bug

After training with diffusers/examples/text_to_image/train_text_to_image_lora.py script
the lora checkpoint couldnt loaded to pipeline

Reproduction

!accelerate launch /content/diffusers/examples/text_to_image/train_text_to_image_lora.py \
 --pretrained_model_name_or_path="stabilityai/stable-diffusion-2-1" \
 --dataset_name="dataset_path" \
 --resolution=512 --center_crop \
 --train_batch_size=2 \
 --gradient_accumulation_steps=1 \
 --dataloader_num_workers=8 \
 --num_train_epochs=200 \
 --max_train_steps=10000 \
 --learning_rate=1e-05 \
 --max_grad_norm=1 \
 --lr_scheduler="cosine" \
 --lr_warmup_steps=100 \
 --checkpointing_steps=500 \
 --allow_tf32 \
 --seed=42 \
 --caption_column="caption" \
 --rank=32 \
 --output_dir="/content/drive/MyDrive/"

import torch
from diffusers import DiffusionPipeline, DDPMScheduler

pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1", use_safetensors=True)
pipe.scheduler = DDPMScheduler.from_config(pipe.scheduler.config)
pipe.load_lora_weights("training_path/checkpoint-1000", weight_name="model.safetensors")
pipe.to("cuda")

[/usr/local/lib/python3.10/dist-packages/diffusers/loaders/lora.py](https://localhost:8080/#) in load_lora_weights(self, pretrained_model_name_or_path_or_dict, adapter_name, **kwargs)
 108 is_correct_format = all("lora" in key for key in state_dict.keys())
 109 if not is_correct_format:
--> 110 raise ValueError("Invalid LoRA checkpoint.")
 111 
 112 low_cpu_mem_usage = kwargs.pop("low_cpu_mem_usage", _LOW_CPU_MEM_USAGE_DEFAULT)

ValueError: Invalid LoRA checkpoint.

Logs

No response

System Info

• diffusers version: 0.25.0.dev0
• Platform: Linux-6.1.58+-x86_64-with-glibc2.35
• Python version: 3.10.12
• PyTorch version (GPU?): 2.1.0+cu121 (True)
• Huggingface_hub version: 0.19.4
• Transformers version: 4.35.2
• Accelerate version: 0.25.0
• xFormers version: not installed
• Using GPU in script?:
• Using distributed or parallel set-up in script?:

Who can help?

@sayakpaul @patrickvonplaten

[sayakpaul]

Could you try loading the checkpoint by providing its absolute path?

Also do keep #6225 in mind for additional implementation correctness.

[maliozer]

@sayakpaul do you mean we need to show exact lora weights safetensors location as parameter for pipe.load_lora_weights() ? ,
/content/drive/MyDrive/research/sdf_ckpt/t2i/sd21_r32/checkpoint-10500/pytorch_lora_weights.safetensors

[sayakpaul]

Yeah.

If that doesn't solve the problem, feel free to share your Colab and I am happy to take a look.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.