Flux Lora Unloading Issue Discussion

[CyberVy]

This issue is for discussing the behavior of FluxLoraLoaderMixin when unloading one or multiple specific LoRA like black-forest-labs/FLUX.1-Canny-dev-lora which expands the shapes before submitting a PR.

Related: #9325 #10989 #10397

[CyberVy]

We considered that users could completely unload a specific LoRA from a model (delete_adapters() doesn't completely unload). See the discussions here:
#9325 (comment). We ultimately decided not to pursue the feature as it's better to keep the LoRA layers when dealing with multiple LoRA layers.

@sayakpaul
Thanks for clarification of the background.

According to #9325 , delete_adapters doesn't completely unload the specific LoRA, because the LoRA layers are still there.

However, delete_adapters can release the RAM/VRAM occupied by specific LoRA, allowing the inference speed to return to its initial level without them. I think it's enough.

What I want to do is to allow FluxLoraLoaderMixin.delete_adapters to correctly unload (even though not completely) a LoRA like black-forest-labs/FLUX.1-Canny-dev-lora which expands the shapes. And make the methods like LoraBaseMixin.enable_lora work properly for those LoRA.

[sayakpaul]

Alrighty then! Let's then jam on the PRs as we discussed above 🎸

[sayakpaul]

@CyberVy just checking if you still want to contribute the PR :)

[CyberVy]

I've finished the code, however I was very busy those days. I'll open a PR after code test in these days.

[sayakpaul]

Thanks! We also buttoned up the #10990 and it's compatible with enable_model_cpu_offload() too. Feel free to test and give feedback.

[CyberVy]

👍 Nice job!
I'll open the PR after #10990 got merged to avoid the possible conflict.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.