diff --git a/docs/source/en/quantization/torchao.md b/docs/source/en/quantization/torchao.md index a501c4efc43a..aa415dbc36e2 100644 --- a/docs/source/en/quantization/torchao.md +++ b/docs/source/en/quantization/torchao.md @@ -83,25 +83,6 @@ Refer to this [table](https://github.com/huggingface/diffusers/pull/10009#issue- > [!TIP] > The FP8 post-training quantization schemes in torchao are effective for GPUs with compute capability of at least 8.9 (RTX-4090, Hopper, etc.). FP8 often provides the best speed, memory, and quality trade-off when generating images and videos. We recommend combining FP8 and torch.compile if your GPU is compatible. -## autoquant - -torchao provides [autoquant](https://docs.pytorch.org/ao/stable/generated/torchao.quantization.autoquant.html#torchao.quantization.autoquant) an automatic quantization API. Autoquantization chooses the best quantization strategy by comparing the performance of each strategy on chosen input types and shapes. This is only supported in Diffusers for individual models at the moment. - -```py -import torch -from diffusers import DiffusionPipeline -from torchao.quantization import autoquant - -# Load the pipeline -pipeline = DiffusionPipeline.from_pretrained( - "black-forest-labs/FLUX.1-schnell", - torch_dtype=torch.bfloat16, - device_map="cuda" -) - -transformer = autoquant(pipeline.transformer) -``` - ## Supported quantization types torchao supports weight-only quantization and weight and dynamic-activation quantization for int8, float3-float8, and uint1-uint7. diff --git a/src/diffusers/quantizers/quantization_config.py b/src/diffusers/quantizers/quantization_config.py index cc5b5cc93d82..c905a928c79d 100644 --- a/src/diffusers/quantizers/quantization_config.py +++ b/src/diffusers/quantizers/quantization_config.py @@ -623,7 +623,7 @@ def _get_torchao_quant_type_to_method(cls): """ if is_torchao_available(): - # TODO(aryan): Support autoquant and sparsify + # TODO(aryan): Support sparsify from torchao.quantization import ( float8_dynamic_activation_float8_weight, float8_static_activation_float8_weight, diff --git a/src/diffusers/quantizers/torchao/torchao_quantizer.py b/src/diffusers/quantizers/torchao/torchao_quantizer.py index 0405afdaaea0..11435b85eb4d 100644 --- a/src/diffusers/quantizers/torchao/torchao_quantizer.py +++ b/src/diffusers/quantizers/torchao/torchao_quantizer.py @@ -344,7 +344,6 @@ def get_cuda_warm_up_factor(self): from torchao.core.config import AOBaseConfig quant_type = self.quantization_config.quant_type - # For autoquant case, it will be treated in the string implementation below in map_to_target_dtype if isinstance(quant_type, AOBaseConfig): # Extract size digit using fuzzy match on the class name config_name = quant_type.__class__.__name__