Is your feature request related to a problem? Please describe.
Bitsandbytes now has basic support for the Apple MPS backend, as I can tell by bitsandbytes-foundation/bitsandbytes#1818 and
bitsandbytes-foundation/bitsandbytes#1875.
The issue is that diffusers does not allow me to use the quantization on Apple hardware, because of the error No GPU found. A GPU is needed for quantization. from here
|
if not (torch.cuda.is_available() or torch.xpu.is_available()): |
|
raise RuntimeError("No GPU found. A GPU is needed for quantization.") |
.
Describe the solution you'd like.
I modified the above lines to
if not (torch.cuda.is_available() or torch.xpu.is_available() or torch.mps.is_available()):
raise RuntimeError("No GPU found. A GPU is needed for quantization.")and tested the change with the quantized version of FLUX.2-dev as described in https://github.com/black-forest-labs/flux2/blob/main/docs/flux2_dev_hf.md#4-bit-transformer-and-4-bit-text-encoder-20g-of-vram, and all worked fine.
I am wondering if that solution could be adopted in the repository. I can open a PR if that is the case.
Describe alternatives you've considered.
None.
Additional context.
Add any other context or screenshots about the feature request here.
Is your feature request related to a problem? Please describe.
Bitsandbytes now has basic support for the Apple MPS backend, as I can tell by bitsandbytes-foundation/bitsandbytes#1818 and
bitsandbytes-foundation/bitsandbytes#1875.
The issue is that diffusers does not allow me to use the quantization on Apple hardware, because of the error
No GPU found. A GPU is needed for quantization.from herediffusers/src/diffusers/quantizers/bitsandbytes/bnb_quantizer.py
Lines 64 to 65 in f2be8bd
Describe the solution you'd like.
I modified the above lines to
and tested the change with the quantized version of FLUX.2-dev as described in https://github.com/black-forest-labs/flux2/blob/main/docs/flux2_dev_hf.md#4-bit-transformer-and-4-bit-text-encoder-20g-of-vram, and all worked fine.
I am wondering if that solution could be adopted in the repository. I can open a PR if that is the case.
Describe alternatives you've considered.
None.
Additional context.
Add any other context or screenshots about the feature request here.