Is there an existing issue for this?
Contact Details
No response
What should this feature add?
Now that Unsloth GGUFs have been rolled out for the quantized versions of Flux Krea and Flux Kontext, you may wish to standardize the original Flux Quantized models on GGUF q4_k_m as well. This will be of particular benefit for users with GPUs that are incompatible with BitsandBytes, such as MPS (OSX) and AMD ROCm.
The relevant Unsloth model links are:
Alternatives
You may also wish to consider moving back to BitsandBytes once support for MPS and ROCm architectures, both of which are planned, matures. BnB should be faster than GGUF and may also use less VRAM. The NF4v2 Flux Dev checkpoint for Forge runs with as little as 6GB of VRAM.
Additional Content
No response
Is there an existing issue for this?
Contact Details
No response
What should this feature add?
Now that Unsloth GGUFs have been rolled out for the quantized versions of Flux Krea and Flux Kontext, you may wish to standardize the original Flux Quantized models on GGUF q4_k_m as well. This will be of particular benefit for users with GPUs that are incompatible with BitsandBytes, such as MPS (OSX) and AMD ROCm.
The relevant Unsloth model links are:
Alternatives
You may also wish to consider moving back to BitsandBytes once support for MPS and ROCm architectures, both of which are planned, matures. BnB should be faster than GGUF and may also use less VRAM. The NF4v2 Flux Dev checkpoint for Forge runs with as little as 6GB of VRAM.
Additional Content
No response