hunyuan_image model/pipeline review

[hlky]

hunyuan_image model/pipeline review

Commit tested: 0f1abc4ae8b0eb2a3b40e82a310507281144c423

Review performed against the repository review rules.

Duplicate-search status: searched hunyuan_image, HunyuanImagePipeline, AutoencoderKLHunyuanImage, AutoencoderKLHunyuanImageRefiner, plus targeted tiling/shortcut/shape terms. Broad matches were the original integration PR #12333 and HunyuanImage 3.0 request #12412; I did not find duplicates for the issues below. A second gh search batch hit GitHub API rate limits, so I completed targeted duplicate checks with web search.

Issue 1: Base VAE default scaling factor breaks pipeline decode

Affected code:

diffusers/src/diffusers/models/autoencoders/autoencoder_kl_hunyuanimage.py

Lines 423 to 465 in 0f1abc4

	@register_to_config
	def __init__(
	self,
	in_channels: int,
	out_channels: int,
	latent_channels: int,
	block_out_channels: tuple[int, ...],
	layers_per_block: int,
	spatial_compression_ratio: int,
	sample_size: int,
	scaling_factor: float = None,
	downsample_match_channel: bool = True,
	upsample_match_channel: bool = True,
	) -> None:
	# fmt: on
	super().__init__()
	self.encoder = HunyuanImageEncoder2D(
	in_channels=in_channels,
	z_channels=latent_channels,
	block_out_channels=block_out_channels,
	num_res_blocks=layers_per_block,
	spatial_compression_ratio=spatial_compression_ratio,
	downsample_match_channel=downsample_match_channel,
	)
	self.decoder = HunyuanImageDecoder2D(
	z_channels=latent_channels,
	out_channels=out_channels,
	block_out_channels=list(reversed(block_out_channels)),
	num_res_blocks=layers_per_block,
	spatial_compression_ratio=spatial_compression_ratio,
	upsample_match_channel=upsample_match_channel,
	)
	# Tiling and slicing configuration
	self.use_slicing = False
	self.use_tiling = False
	# Tiling parameters
	self.tile_sample_min_size = sample_size
	self.tile_latent_min_size = sample_size // spatial_compression_ratio
	self.tile_overlap_factor = 0.25

diffusers/src/diffusers/pipelines/hunyuan_image/pipeline_hunyuanimage.py

Lines 855 to 860 in 0f1abc4

	if output_type == "latent":
	image = latents
	else:
	latents = latents.to(self.vae.dtype) / self.vae.config.scaling_factor
	image = self.vae.decode(latents, return_dict=False)[0]
	image = self.image_processor.postprocess(image, output_type=output_type)

Problem:
AutoencoderKLHunyuanImage defaults scaling_factor to None, but HunyuanImagePipeline always divides latents by self.vae.config.scaling_factor before decoding.

Impact:
A pipeline assembled from default tiny components can denoise successfully and then fail at decode with TypeError. The refiner VAE already has a numeric default.

Reproduction:

import torch
from diffusers import AutoencoderKLHunyuanImage, FlowMatchEulerDiscreteScheduler, HunyuanImagePipeline, HunyuanImageTransformer2DModel

transformer = HunyuanImageTransformer2DModel(
 in_channels=4, out_channels=4, num_attention_heads=1, attention_head_dim=4,
 num_layers=0, num_single_layers=0, num_refiner_layers=0,
 patch_size=(1, 1), text_embed_dim=4, text_embed_2_dim=4, rope_axes_dim=(2, 2),
)
vae = AutoencoderKLHunyuanImage(
 in_channels=3, out_channels=3, latent_channels=4, block_out_channels=(32,),
 layers_per_block=1, spatial_compression_ratio=8, sample_size=32,
)
pipe = HunyuanImagePipeline(
 scheduler=FlowMatchEulerDiscreteScheduler(), vae=vae, text_encoder=None, tokenizer=None,
 text_encoder_2=None, tokenizer_2=None, transformer=transformer, guider=None, ocr_guider=None,
)
pipe(
 prompt=None,
 prompt_embeds=torch.randn(1, 1, 4),
 prompt_embeds_mask=torch.ones(1, 1, dtype=torch.long),
 prompt_embeds_2=torch.zeros(1, 1, 4),
 prompt_embeds_mask_2=torch.zeros(1, 1, dtype=torch.long),
 height=16, width=16, num_inference_steps=1, output_type="pt",
)

Relevant precedent:

diffusers/src/diffusers/models/autoencoders/autoencoder_kl_hunyuanimage_refiner.py

Lines 604 to 617 in 0f1abc4

	@register_to_config
	def __init__(
	self,
	in_channels: int = 3,
	out_channels: int = 3,
	latent_channels: int = 32,
	block_out_channels: tuple[int, ...] = (128, 256, 512, 1024, 1024),
	layers_per_block: int = 2,
	spatial_compression_ratio: int = 16,
	temporal_compression_ratio: int = 4,
	downsample_match_channel: bool = True,
	upsample_match_channel: bool = True,
	scaling_factor: float = 1.03682,
	) -> None:

Suggested fix:

scaling_factor: float = 0.476986

Issue 2: HunyuanImagePipelineOutput is not exported from the pipeline package

Affected code:

diffusers/src/diffusers/pipelines/hunyuan_image/__init__.py

Lines 25 to 37 in 0f1abc4

	_import_structure["pipeline_hunyuanimage"] = ["HunyuanImagePipeline"]
	_import_structure["pipeline_hunyuanimage_refiner"] = ["HunyuanImageRefinerPipeline"]
	if TYPE_CHECKING or DIFFUSERS_SLOW_IMPORT:
	try:
	if not (is_transformers_available() and is_torch_available()):
	raise OptionalDependencyNotAvailable()
	except OptionalDependencyNotAvailable:
	from ...utils.dummy_torch_and_transformers_objects import *
	else:
	from .pipeline_hunyuanimage import HunyuanImagePipeline
	from .pipeline_hunyuanimage_refiner import HunyuanImageRefinerPipeline

diffusers/src/diffusers/pipelines/hunyuan_image/pipeline_output.py

Lines 10 to 20 in 0f1abc4

	class HunyuanImagePipelineOutput(BaseOutput):
	"""
	Output class for HunyuanImage pipelines.
	Args:
	images (`list[PIL.Image.Image]` or `np.ndarray`)
	List of denoised PIL images of length `batch_size` or numpy array of shape `(batch_size, height, width,
	num_channels)`. PIL images or numpy array present the denoised images of the diffusion pipeline.
	"""
	images: list[PIL.Image.Image, np.ndarray]

Problem:
The output class exists and is documented, but diffusers.pipelines.hunyuan_image only lazy-exports the two pipelines.

Impact:
Users cannot import the output type from the package namespace, unlike similar pipeline families.

Reproduction:

from diffusers.pipelines.hunyuan_image import HunyuanImagePipelineOutput

Relevant precedent:

diffusers/src/diffusers/pipelines/qwenimage/__init__.py

Lines 15 to 43 in 0f1abc4

	_import_structure = {"pipeline_output": ["QwenImagePipelineOutput", "QwenImagePriorReduxPipelineOutput"]}
	try:
	if not (is_transformers_available() and is_torch_available()):
	raise OptionalDependencyNotAvailable()
	except OptionalDependencyNotAvailable:
	from ...utils import dummy_torch_and_transformers_objects # noqa F403
	_dummy_objects.update(get_objects_from_module(dummy_torch_and_transformers_objects))
	else:
	_import_structure["modeling_qwenimage"] = ["ReduxImageEncoder"]
	_import_structure["pipeline_qwenimage"] = ["QwenImagePipeline"]
	_import_structure["pipeline_qwenimage_controlnet"] = ["QwenImageControlNetPipeline"]
	_import_structure["pipeline_qwenimage_controlnet_inpaint"] = ["QwenImageControlNetInpaintPipeline"]
	_import_structure["pipeline_qwenimage_edit"] = ["QwenImageEditPipeline"]
	_import_structure["pipeline_qwenimage_edit_inpaint"] = ["QwenImageEditInpaintPipeline"]
	_import_structure["pipeline_qwenimage_edit_plus"] = ["QwenImageEditPlusPipeline"]
	_import_structure["pipeline_qwenimage_img2img"] = ["QwenImageImg2ImgPipeline"]
	_import_structure["pipeline_qwenimage_inpaint"] = ["QwenImageInpaintPipeline"]
	_import_structure["pipeline_qwenimage_layered"] = ["QwenImageLayeredPipeline"]
	if TYPE_CHECKING or DIFFUSERS_SLOW_IMPORT:
	try:
	if not (is_transformers_available() and is_torch_available()):
	raise OptionalDependencyNotAvailable()
	except OptionalDependencyNotAvailable:
	from ...utils.dummy_torch_and_transformers_objects import * # noqa F403
	else:
	from .pipeline_qwenimage import QwenImagePipeline

Suggested fix:

_import_structure["pipeline_output"] = ["HunyuanImagePipelineOutput"]
...
from .pipeline_output import HunyuanImagePipelineOutput

Issue 3: Base VAE tiled encode crashes on 4D image tensors

Affected code:

diffusers/src/diffusers/models/autoencoders/autoencoder_kl_hunyuanimage.py

Lines 582 to 620 in 0f1abc4

	def tiled_encode(self, x: torch.Tensor) -> torch.Tensor:
	"""
	Encode input using spatial tiling strategy.
	Args:
	x (`torch.Tensor`): Input tensor of shape (B, C, T, H, W).
	Returns:
	`torch.Tensor`:
	The latent representation of the encoded images.
	"""
	_, _, _, height, width = x.shape
	overlap_size = int(self.tile_sample_min_size * (1 - self.tile_overlap_factor))
	blend_extent = int(self.tile_latent_min_size * self.tile_overlap_factor)
	row_limit = self.tile_latent_min_size - blend_extent
	rows = []
	for i in range(0, height, overlap_size):
	row = []
	for j in range(0, width, overlap_size):
	tile = x[:, :, :, i : i + self.tile_sample_min_size, j : j + self.tile_sample_min_size]
	tile = self.encoder(tile)
	row.append(tile)
	rows.append(row)
	result_rows = []
	for i, row in enumerate(rows):
	result_row = []
	for j, tile in enumerate(row):
	if i > 0:
	tile = self.blend_v(rows[i - 1][j], tile, blend_extent)
	if j > 0:
	tile = self.blend_h(row[j - 1], tile, blend_extent)
	result_row.append(tile[:, :, :, :row_limit, :row_limit])
	result_rows.append(torch.cat(result_row, dim=-1))
	moments = torch.cat(result_rows, dim=-2)
	return moments

Problem:
AutoencoderKLHunyuanImage is a 2D image VAE, but tiled_encode() unpacks x.shape as if it were 5D and slices tiles with an extra temporal dimension.

Impact:
vae.enable_tiling(); vae.encode(image) fails for image tensors large enough to trigger tiling. Existing tests cover tiled decode through the pipeline, but not tiled encode.

Reproduction:

import torch
from diffusers import AutoencoderKLHunyuanImage

vae = AutoencoderKLHunyuanImage(
 in_channels=3, out_channels=3, latent_channels=4, block_out_channels=(32,),
 layers_per_block=1, spatial_compression_ratio=1, sample_size=8,
)
vae.enable_tiling(tile_sample_min_size=4)
vae.encode(torch.randn(1, 3, 8, 8))

Relevant precedent:
The same file's tiled_decode() correctly treats base VAE latents as 4D:

diffusers/src/diffusers/models/autoencoders/autoencoder_kl_hunyuanimage.py

Lines 622 to 664 in 0f1abc4

	def tiled_decode(self, z: torch.Tensor, return_dict: bool = True) -> DecoderOutput | torch.Tensor:
	"""
	Decode latent using spatial tiling strategy.
	Args:
	z (`torch.Tensor`): Latent tensor of shape (B, C, H, W).
	return_dict (`bool`, *optional*, defaults to `True`):
	Whether or not to return a [`~models.vae.DecoderOutput`] instead of a plain tuple.
	Returns:
	[`~models.vae.DecoderOutput`] or `tuple`:
	If return_dict is True, a [`~models.vae.DecoderOutput`] is returned, otherwise a plain `tuple` is
	returned.
	"""
	_, _, height, width = z.shape
	overlap_size = int(self.tile_latent_min_size * (1 - self.tile_overlap_factor))
	blend_extent = int(self.tile_sample_min_size * self.tile_overlap_factor)
	row_limit = self.tile_sample_min_size - blend_extent
	rows = []
	for i in range(0, height, overlap_size):
	row = []
	for j in range(0, width, overlap_size):
	tile = z[:, :, i : i + self.tile_latent_min_size, j : j + self.tile_latent_min_size]
	decoded = self.decoder(tile)
	row.append(decoded)
	rows.append(row)
	result_rows = []
	for i, row in enumerate(rows):
	result_row = []
	for j, tile in enumerate(row):
	if i > 0:
	tile = self.blend_v(rows[i - 1][j], tile, blend_extent)
	if j > 0:
	tile = self.blend_h(row[j - 1], tile, blend_extent)
	result_row.append(tile[:, :, :row_limit, :row_limit])
	result_rows.append(torch.cat(result_row, dim=-1))
	dec = torch.cat(result_rows, dim=-2)
	if not return_dict:
	return (dec,)
	return DecoderOutput(sample=dec)

Suggested fix:

_, _, height, width = x.shape
...
tile = x[:, :, i : i + self.tile_sample_min_size, j : j + self.tile_sample_min_size]
...
result_row.append(tile[:, :, :row_limit, :row_limit])

Issue 4: Refiner VAE tiling mixes sample and latent units

Affected code:

diffusers/src/diffusers/models/autoencoders/autoencoder_kl_hunyuanimage_refiner.py

Lines 793 to 841 in 0f1abc4

	def tiled_encode(self, x: torch.Tensor) -> torch.Tensor:
	r"""Encode a batch of images using a tiled encoder.
	Args:
	x (`torch.Tensor`): Input batch of videos.
	Returns:
	`torch.Tensor`:
	The latent representation of the encoded videos.
	"""
	_, _, _, height, width = x.shape
	tile_latent_min_height = self.tile_sample_min_height // self.spatial_compression_ratio
	tile_latent_min_width = self.tile_sample_min_width // self.spatial_compression_ratio
	overlap_height = int(tile_latent_min_height * (1 - self.tile_overlap_factor)) # 256 * (1 - 0.25) = 192
	overlap_width = int(tile_latent_min_width * (1 - self.tile_overlap_factor)) # 256 * (1 - 0.25) = 192
	blend_height = int(tile_latent_min_height * self.tile_overlap_factor) # 8 * 0.25 = 2
	blend_width = int(tile_latent_min_width * self.tile_overlap_factor) # 8 * 0.25 = 2
	row_limit_height = tile_latent_min_height - blend_height # 8 - 2 = 6
	row_limit_width = tile_latent_min_width - blend_width # 8 - 2 = 6
	rows = []
	for i in range(0, height, overlap_height):
	row = []
	for j in range(0, width, overlap_width):
	tile = x[
	:,
	:,
	:,
	i : i + self.tile_sample_min_height,
	j : j + self.tile_sample_min_width,
	]
	tile = self.encoder(tile)
	row.append(tile)
	rows.append(row)
	result_rows = []
	for i, row in enumerate(rows):
	result_row = []
	for j, tile in enumerate(row):
	if i > 0:
	tile = self.blend_v(rows[i - 1][j], tile, blend_height)
	if j > 0:
	tile = self.blend_h(row[j - 1], tile, blend_width)
	result_row.append(tile[:, :, :, :row_limit_height, :row_limit_width])
	result_rows.append(torch.cat(result_row, dim=-1))
	moments = torch.cat(result_rows, dim=-2)
	return moments

diffusers/src/diffusers/models/autoencoders/autoencoder_kl_hunyuanimage_refiner.py

Lines 843 to 895 in 0f1abc4

	def tiled_decode(self, z: torch.Tensor) -> torch.Tensor:
	r"""
	Decode a batch of images using a tiled decoder.
	Args:
	z (`torch.Tensor`): Input batch of latent vectors.
	return_dict (`bool`, *optional*, defaults to `True`):
	Whether or not to return a [`~models.vae.DecoderOutput`] instead of a plain tuple.
	Returns:
	[`~models.vae.DecoderOutput`] or `tuple`:
	If return_dict is True, a [`~models.vae.DecoderOutput`] is returned, otherwise a plain `tuple` is
	returned.
	"""
	_, _, _, height, width = z.shape
	tile_latent_min_height = self.tile_sample_min_height // self.spatial_compression_ratio
	tile_latent_min_width = self.tile_sample_min_width // self.spatial_compression_ratio
	overlap_height = int(tile_latent_min_height * (1 - self.tile_overlap_factor)) # 8 * (1 - 0.25) = 6
	overlap_width = int(tile_latent_min_width * (1 - self.tile_overlap_factor)) # 8 * (1 - 0.25) = 6
	blend_height = int(tile_latent_min_height * self.tile_overlap_factor) # 256 * 0.25 = 64
	blend_width = int(tile_latent_min_width * self.tile_overlap_factor) # 256 * 0.25 = 64
	row_limit_height = tile_latent_min_height - blend_height # 256 - 64 = 192
	row_limit_width = tile_latent_min_width - blend_width # 256 - 64 = 192
	rows = []
	for i in range(0, height, overlap_height):
	row = []
	for j in range(0, width, overlap_width):
	tile = z[
	:,
	:,
	:,
	i : i + tile_latent_min_height,
	j : j + tile_latent_min_width,
	]
	decoded = self.decoder(tile)
	row.append(decoded)
	rows.append(row)
	result_rows = []
	for i, row in enumerate(rows):
	result_row = []
	for j, tile in enumerate(row):
	if i > 0:
	tile = self.blend_v(rows[i - 1][j], tile, blend_height)
	if j > 0:
	tile = self.blend_h(row[j - 1], tile, blend_width)
	result_row.append(tile[:, :, :, :row_limit_height, :row_limit_width])
	result_rows.append(torch.cat(result_row, dim=-1))
	dec = torch.cat(result_rows, dim=-2)
	return dec

Problem:
tiled_encode() iterates over sample-space height/width using latent-space overlap values. tiled_decode() crops decoded sample tiles using latent-space row limits.

Impact:
Tiled encode can generate invalid edge tiles, and tiled decode returns the wrong spatial size.

Reproduction:

import torch
from diffusers import AutoencoderKLHunyuanImageRefiner

vae = AutoencoderKLHunyuanImageRefiner(
 in_channels=3, out_channels=3, latent_channels=4, block_out_channels=(8, 8),
 layers_per_block=0, spatial_compression_ratio=2, temporal_compression_ratio=1,
)
z = torch.randn(1, 4, 1, 8, 8)
plain = vae.decode(z).sample.shape
vae.enable_tiling(tile_sample_min_height=8, tile_sample_min_width=8, tile_overlap_factor=0.25)
tiled = vae.decode(z).sample.shape
print(plain, tiled) # plain is [1, 3, 1, 16, 16], tiled is [1, 3, 1, 9, 9]

Relevant precedent:

diffusers/src/diffusers/models/autoencoders/autoencoder_kl_hunyuanvideo15.py

Lines 829 to 899 in 0f1abc4

	def tiled_encode(self, x: torch.Tensor) -> torch.Tensor:
	r"""Encode a batch of images using a tiled encoder.
	Args:
	x (`torch.Tensor`): Input batch of videos.
	Returns:
	`torch.Tensor`:
	The latent representation of the encoded videos.
	"""
	_, _, _, height, width = x.shape
	overlap_height = int(self.tile_sample_min_height * (1 - self.tile_overlap_factor)) # 256 * (1 - 0.25) = 192
	overlap_width = int(self.tile_sample_min_width * (1 - self.tile_overlap_factor)) # 256 * (1 - 0.25) = 192
	blend_height = int(self.tile_latent_min_height * self.tile_overlap_factor) # 8 * 0.25 = 2
	blend_width = int(self.tile_latent_min_width * self.tile_overlap_factor) # 8 * 0.25 = 2
	row_limit_height = self.tile_latent_min_height - blend_height # 8 - 2 = 6
	row_limit_width = self.tile_latent_min_width - blend_width # 8 - 2 = 6
	rows = []
	for i in range(0, height, overlap_height):
	row = []
	for j in range(0, width, overlap_width):
	tile = x[
	:,
	:,
	:,
	i : i + self.tile_sample_min_height,
	j : j + self.tile_sample_min_width,
	]
	tile = self.encoder(tile)
	row.append(tile)
	rows.append(row)
	result_rows = []
	for i, row in enumerate(rows):
	result_row = []
	for j, tile in enumerate(row):
	if i > 0:
	tile = self.blend_v(rows[i - 1][j], tile, blend_height)
	if j > 0:
	tile = self.blend_h(row[j - 1], tile, blend_width)
	result_row.append(tile[:, :, :, :row_limit_height, :row_limit_width])
	result_rows.append(torch.cat(result_row, dim=-1))
	moments = torch.cat(result_rows, dim=-2)
	return moments
	def tiled_decode(self, z: torch.Tensor) -> torch.Tensor:
	r"""
	Decode a batch of images using a tiled decoder.
	Args:
	z (`torch.Tensor`): Input batch of latent vectors.
	return_dict (`bool`, *optional*, defaults to `True`):
	Whether or not to return a [`~models.vae.DecoderOutput`] instead of a plain tuple.
	Returns:
	[`~models.vae.DecoderOutput`] or `tuple`:
	If return_dict is True, a [`~models.vae.DecoderOutput`] is returned, otherwise a plain `tuple` is
	returned.
	"""
	_, _, _, height, width = z.shape
	overlap_height = int(self.tile_latent_min_height * (1 - self.tile_overlap_factor)) # 8 * (1 - 0.25) = 6
	overlap_width = int(self.tile_latent_min_width * (1 - self.tile_overlap_factor)) # 8 * (1 - 0.25) = 6
	blend_height = int(self.tile_sample_min_height * self.tile_overlap_factor) # 256 * 0.25 = 64
	blend_width = int(self.tile_sample_min_width * self.tile_overlap_factor) # 256 * 0.25 = 64
	row_limit_height = self.tile_sample_min_height - blend_height # 256 - 64 = 192
	row_limit_width = self.tile_sample_min_width - blend_width # 256 - 64 = 192

Suggested fix:
Use sample-space stride/overlap for encode iteration, latent-space crop sizes for encoded tiles, latent-space stride for decode iteration, and sample-space crop sizes for decoded tiles. The existing tile_sample_stride_height / tile_sample_stride_width fields should either be used or removed.

Issue 5: Base VAE shortcut projection is applied to the wrong tensor

Affected code:

diffusers/src/diffusers/models/autoencoders/autoencoder_kl_hunyuanimage.py

Lines 61 to 77 in 0f1abc4

	def forward(self, x):
	# Apply shortcut connection
	residual = x
	# First normalization and activation
	x = self.norm1(x)
	x = self.nonlinearity(x)
	x = self.conv1(x)
	x = self.norm2(x)
	x = self.nonlinearity(x)
	x = self.conv2(x)
	if self.conv_shortcut is not None:
	x = self.conv_shortcut(x)
	# Add residual connection
	return x + residual

Problem:
When in_channels != out_channels, HunyuanImageResnetBlock.forward() applies conv_shortcut to x after conv2, not to the saved residual.

Impact:
Non-default serialized configs with downsample_match_channel=False or upsample_match_channel=False crash when a resnet block changes channels.

Reproduction:

import torch
from diffusers import AutoencoderKLHunyuanImage

vae = AutoencoderKLHunyuanImage(
 in_channels=3, out_channels=3, latent_channels=4, block_out_channels=(32, 64),
 layers_per_block=1, spatial_compression_ratio=2, sample_size=16,
 downsample_match_channel=False,
)
vae.encode(torch.randn(1, 3, 16, 16))

Relevant precedent:

diffusers/src/diffusers/models/autoencoders/autoencoder_kl_hunyuanimage_refiner.py

Lines 253 to 267 in 0f1abc4

	def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
	residual = hidden_states
	hidden_states = self.norm1(hidden_states)
	hidden_states = self.nonlinearity(hidden_states)
	hidden_states = self.conv1(hidden_states)
	hidden_states = self.norm2(hidden_states)
	hidden_states = self.nonlinearity(hidden_states)
	hidden_states = self.conv2(hidden_states)
	if self.conv_shortcut is not None:
	residual = self.conv_shortcut(residual)
	return hidden_states + residual

Suggested fix:

if self.conv_shortcut is not None:
 residual = self.conv_shortcut(residual)
return x + residual

Issue 6: Transformer silently truncates latent sizes not divisible by patch_size

Affected code:

diffusers/src/diffusers/models/transformers/transformer_hunyuanimage.py

Lines 766 to 885 in 0f1abc4

	post_patch_sizes = tuple(d // p for d, p in zip(sizes, self.config.patch_size))
	# 1. RoPE
	image_rotary_emb = self.rope(hidden_states)
	# 2. Conditional embeddings
	encoder_attention_mask = encoder_attention_mask.bool()
	temb = self.time_guidance_embed(timestep, guidance=guidance, timestep_r=timestep_r)
	hidden_states = self.x_embedder(hidden_states)
	encoder_hidden_states = self.context_embedder(encoder_hidden_states, timestep, encoder_attention_mask)
	if self.context_embedder_2 is not None and encoder_hidden_states_2 is not None:
	encoder_hidden_states_2 = self.context_embedder_2(encoder_hidden_states_2)
	encoder_attention_mask_2 = encoder_attention_mask_2.bool()
	# reorder and combine text tokens: combine valid tokens first, then padding
	new_encoder_hidden_states = []
	new_encoder_attention_mask = []
	for text, text_mask, text_2, text_mask_2 in zip(
	encoder_hidden_states, encoder_attention_mask, encoder_hidden_states_2, encoder_attention_mask_2
	):
	# Concatenate: [valid_mllm, valid_byt5, invalid_mllm, invalid_byt5]
	new_encoder_hidden_states.append(
	torch.cat(
	[
	text_2[text_mask_2], # valid byt5
	text[text_mask], # valid mllm
	text_2[~text_mask_2], # invalid byt5
	text[~text_mask], # invalid mllm
	],
	dim=0,
	)
	)
	# Apply same reordering to attention masks
	new_encoder_attention_mask.append(
	torch.cat(
	[
	text_mask_2[text_mask_2],
	text_mask[text_mask],
	text_mask_2[~text_mask_2],
	text_mask[~text_mask],
	],
	dim=0,
	)
	)
	encoder_hidden_states = torch.stack(new_encoder_hidden_states)
	encoder_attention_mask = torch.stack(new_encoder_attention_mask)
	attention_mask = torch.nn.functional.pad(encoder_attention_mask, (hidden_states.shape[1], 0), value=True)
	attention_mask = attention_mask.unsqueeze(1).unsqueeze(2)
	# 3. Transformer blocks
	if torch.is_grad_enabled() and self.gradient_checkpointing:
	for block in self.transformer_blocks:
	hidden_states, encoder_hidden_states = self._gradient_checkpointing_func(
	block,
	hidden_states,
	encoder_hidden_states,
	temb,
	attention_mask=attention_mask,
	image_rotary_emb=image_rotary_emb,
	)
	for block in self.single_transformer_blocks:
	hidden_states, encoder_hidden_states = self._gradient_checkpointing_func(
	block,
	hidden_states,
	encoder_hidden_states,
	temb,
	attention_mask=attention_mask,
	image_rotary_emb=image_rotary_emb,
	)
	else:
	for block in self.transformer_blocks:
	hidden_states, encoder_hidden_states = block(
	hidden_states,
	encoder_hidden_states,
	temb,
	attention_mask=attention_mask,
	image_rotary_emb=image_rotary_emb,
	)
	for block in self.single_transformer_blocks:
	hidden_states, encoder_hidden_states = block(
	hidden_states,
	encoder_hidden_states,
	temb,
	attention_mask=attention_mask,
	image_rotary_emb=image_rotary_emb,
	)
	# 4. Output projection
	hidden_states = self.norm_out(hidden_states, temb)
	hidden_states = self.proj_out(hidden_states)
	# 5. unpatchify
	# reshape: [batch_size, *post_patch_dims, channels, *patch_size]
	out_channels = self.config.out_channels
	reshape_dims = [batch_size] + list(post_patch_sizes) + [out_channels] + list(self.config.patch_size)
	hidden_states = hidden_states.reshape(*reshape_dims)
	# create permutation pattern: batch, channels, then interleave post_patch and patch dims
	# For 4D: [0, 3, 1, 4, 2, 5] -> batch, channels, post_patch_height, patch_size_height, post_patch_width, patch_size_width
	# For 5D: [0, 4, 1, 5, 2, 6, 3, 7] -> batch, channels, post_patch_frame, patch_size_frame, post_patch_height, patch_size_height, post_patch_width, patch_size_width
	ndim = len(post_patch_sizes)
	permute_pattern = [0, ndim + 1] # batch, channels
	for i in range(ndim):
	permute_pattern.extend([i + 1, ndim + 2 + i]) # post_patch_sizes[i], patch_sizes[i]
	hidden_states = hidden_states.permute(*permute_pattern)
	# flatten patch dimensions: flatten each (post_patch_size, patch_size) pair
	# batch_size, channels, post_patch_sizes[0] * patch_sizes[0], post_patch_sizes[1] * patch_sizes[1], ...
	final_dims = [batch_size, out_channels] + [
	post_patch * patch for post_patch, patch in zip(post_patch_sizes, self.config.patch_size)
	]
	hidden_states = hidden_states.reshape(*final_dims)

diffusers/src/diffusers/pipelines/hunyuan_image/pipeline_hunyuanimage.py

Lines 406 to 486 in 0f1abc4

	if height % (self.vae_scale_factor * 2) != 0 or width % (self.vae_scale_factor * 2) != 0:
	logger.warning(
	f"`height` and `width` have to be divisible by {self.vae_scale_factor * 2} but are {height} and {width}. Dimensions will be resized accordingly"
	)
	if callback_on_step_end_tensor_inputs is not None and not all(
	k in self._callback_tensor_inputs for k in callback_on_step_end_tensor_inputs
	):
	raise ValueError(
	f"`callback_on_step_end_tensor_inputs` has to be in {self._callback_tensor_inputs}, but found {[k for k in callback_on_step_end_tensor_inputs if k not in self._callback_tensor_inputs]}"
	)
	if prompt is not None and prompt_embeds is not None:
	raise ValueError(
	f"Cannot forward both `prompt`: {prompt} and `prompt_embeds`: {prompt_embeds}. Please make sure to"
	" only forward one of the two."
	)
	elif prompt is None and prompt_embeds is None:
	raise ValueError(
	"Provide either `prompt` or `prompt_embeds`. Cannot leave both `prompt` and `prompt_embeds` undefined."
	)
	elif prompt is not None and (not isinstance(prompt, str) and not isinstance(prompt, list)):
	raise ValueError(f"`prompt` has to be of type `str` or `list` but is {type(prompt)}")
	if negative_prompt is not None and negative_prompt_embeds is not None:
	raise ValueError(
	f"Cannot forward both `negative_prompt`: {negative_prompt} and `negative_prompt_embeds`:"
	f" {negative_prompt_embeds}. Please make sure to only forward one of the two."
	)
	if prompt_embeds is not None and prompt_embeds_mask is None:
	raise ValueError(
	"If `prompt_embeds` are provided, `prompt_embeds_mask` also have to be passed. Make sure to generate `prompt_embeds_mask` from the same text encoder that was used to generate `prompt_embeds`."
	)
	if negative_prompt_embeds is not None and negative_prompt_embeds_mask is None:
	raise ValueError(
	"If `negative_prompt_embeds` are provided, `negative_prompt_embeds_mask` also have to be passed. Make sure to generate `negative_prompt_embeds_mask` from the same text encoder that was used to generate `negative_prompt_embeds`."
	)
	if prompt is None and prompt_embeds_2 is None:
	raise ValueError(
	"Provide either `prompt` or `prompt_embeds_2`. Cannot leave both `prompt` and `prompt_embeds_2` undefined."
	)
	if prompt_embeds_2 is not None and prompt_embeds_mask_2 is None:
	raise ValueError(
	"If `prompt_embeds_2` are provided, `prompt_embeds_mask_2` also have to be passed. Make sure to generate `prompt_embeds_mask_2` from the same text encoder that was used to generate `prompt_embeds_2`."
	)
	if negative_prompt_embeds_2 is not None and negative_prompt_embeds_mask_2 is None:
	raise ValueError(
	"If `negative_prompt_embeds_2` are provided, `negative_prompt_embeds_mask_2` also have to be passed. Make sure to generate `negative_prompt_embeds_mask_2` from the same text encoder that was used to generate `negative_prompt_embeds_2`."
	)
	def prepare_latents(
	self,
	batch_size,
	num_channels_latents,
	height,
	width,
	dtype,
	device,
	generator,
	latents=None,
	):
	height = int(height) // self.vae_scale_factor
	width = int(width) // self.vae_scale_factor
	shape = (batch_size, num_channels_latents, height, width)
	if latents is not None:
	return latents.to(device=device, dtype=dtype)
	if isinstance(generator, list) and len(generator) != batch_size:
	raise ValueError(
	f"You have passed a list of generators of length {len(generator)}, but requested an effective batch"
	f" size of {batch_size}. Make sure the batch size matches the length of the generators."
	)
	latents = randn_tensor(shape, generator=generator, device=device, dtype=dtype)
	return latents

diffusers/src/diffusers/pipelines/hunyuan_image/pipeline_hunyuanimage_refiner.py

Lines 295 to 375 in 0f1abc4

	if height % (self.vae_scale_factor * 2) != 0 or width % (self.vae_scale_factor * 2) != 0:
	logger.warning(
	f"`height` and `width` have to be divisible by {self.vae_scale_factor * 2} but are {height} and {width}. Dimensions will be resized accordingly"
	)
	if callback_on_step_end_tensor_inputs is not None and not all(
	k in self._callback_tensor_inputs for k in callback_on_step_end_tensor_inputs
	):
	raise ValueError(
	f"`callback_on_step_end_tensor_inputs` has to be in {self._callback_tensor_inputs}, but found {[k for k in callback_on_step_end_tensor_inputs if k not in self._callback_tensor_inputs]}"
	)
	if prompt is not None and prompt_embeds is not None:
	raise ValueError(
	f"Cannot forward both `prompt`: {prompt} and `prompt_embeds`: {prompt_embeds}. Please make sure to"
	" only forward one of the two."
	)
	elif prompt is None and prompt_embeds is None:
	raise ValueError(
	"Provide either `prompt` or `prompt_embeds`. Cannot leave both `prompt` and `prompt_embeds` undefined."
	)
	elif prompt is not None and (not isinstance(prompt, str) and not isinstance(prompt, list)):
	raise ValueError(f"`prompt` has to be of type `str` or `list` but is {type(prompt)}")
	if negative_prompt is not None and negative_prompt_embeds is not None:
	raise ValueError(
	f"Cannot forward both `negative_prompt`: {negative_prompt} and `negative_prompt_embeds`:"
	f" {negative_prompt_embeds}. Please make sure to only forward one of the two."
	)
	if prompt_embeds is not None and prompt_embeds_mask is None:
	raise ValueError(
	"If `prompt_embeds` are provided, `prompt_embeds_mask` also have to be passed. Make sure to generate `prompt_embeds_mask` from the same text encoder that was used to generate `prompt_embeds`."
	)
	if negative_prompt_embeds is not None and negative_prompt_embeds_mask is None:
	raise ValueError(
	"If `negative_prompt_embeds` are provided, `negative_prompt_embeds_mask` also have to be passed. Make sure to generate `negative_prompt_embeds_mask` from the same text encoder that was used to generate `negative_prompt_embeds`."
	)
	def prepare_latents(
	self,
	image_latents,
	batch_size,
	num_channels_latents,
	height,
	width,
	dtype,
	device,
	generator,
	latents=None,
	strength=0.25,
	):
	height = int(height) // self.vae_scale_factor
	width = int(width) // self.vae_scale_factor
	shape = (batch_size, num_channels_latents, 1, height, width)
	if latents is None:
	latents = randn_tensor(shape, generator=generator, device=device, dtype=dtype)
	else:
	latents = latents.to(device=device, dtype=dtype)
	if batch_size > image_latents.shape[0] and batch_size % image_latents.shape[0] == 0:
	# expand init_latents for batch_size
	additional_image_per_prompt = batch_size // image_latents.shape[0]
	image_latents = torch.cat([image_latents] * additional_image_per_prompt, dim=0)
	elif batch_size > image_latents.shape[0] and batch_size % image_latents.shape[0] != 0:
	raise ValueError(
	f"Cannot duplicate `image` of batch size {image_latents.shape[0]} to {batch_size} text prompts."
	)
	if isinstance(generator, list) and len(generator) != batch_size:
	raise ValueError(
	f"You have passed a list of generators of length {len(generator)}, but requested an effective batch"
	f" size of {batch_size}. Make sure the batch size matches the length of the generators."
	)
	noise = randn_tensor(shape, generator=generator, device=device, dtype=dtype)
	cond_latents = strength * noise + (1 - strength) * image_latents
	return latents, cond_latents

Problem:
The transformer computes post_patch_sizes = d // p and unpatchifies to post_patch * patch, silently dropping remainder pixels. The pipelines warn about divisibility but do not round latent sizes to the transformer patch grid.

Impact:
Custom configs with patch_size > 1 can produce a model output smaller than the input latents, leading to scheduler shape errors or silent shape loss when the model is used directly.

Reproduction:

import torch
from diffusers import HunyuanImageTransformer2DModel

model = HunyuanImageTransformer2DModel(
 in_channels=4, out_channels=4, num_attention_heads=1, attention_head_dim=4,
 num_layers=0, num_single_layers=0, num_refiner_layers=0,
 patch_size=(2, 2), text_embed_dim=4, rope_axes_dim=(2, 2),
)
out = model(
 hidden_states=torch.randn(1, 4, 3, 3),
 timestep=torch.ones(1),
 encoder_hidden_states=torch.randn(1, 1, 4),
 encoder_attention_mask=torch.ones(1, 1, dtype=torch.long),
).sample
print(out.shape) # torch.Size([1, 4, 2, 2])

Relevant precedent:

diffusers/src/diffusers/pipelines/qwenimage/pipeline_qwenimage.py

Lines 346 to 348 in 0f1abc4

	# latent height and width to be divisible by 2.
	height = 2 * (int(height) // (vae_scale_factor * 2))
	width = 2 * (int(width) // (vae_scale_factor * 2))

Suggested fix:

if any(size % patch != 0 for size, patch in zip(sizes, self.config.patch_size)):
 raise ValueError(f"`hidden_states` spatial/temporal sizes {sizes} must be divisible by patch_size {self.config.patch_size}.")

Issue 7: Base VAE has checkpointing code but disables the public capability flag

Affected code:

diffusers/src/diffusers/models/autoencoders/autoencoder_kl_hunyuanimage.py

Lines 296 to 308 in 0f1abc4

	def forward(self, x: torch.Tensor) -> torch.Tensor:
	x = self.conv_in(x)
	## downsamples
	for down_block in self.down_blocks:
	if torch.is_grad_enabled() and self.gradient_checkpointing:
	x = self._gradient_checkpointing_func(down_block, x)
	else:
	x = down_block(x)
	## middle
	if torch.is_grad_enabled() and self.gradient_checkpointing:
	x = self._gradient_checkpointing_func(self.mid_block, x)

diffusers/src/diffusers/models/autoencoders/autoencoder_kl_hunyuanimage.py

Lines 393 to 403 in 0f1abc4

	def forward(self, x: torch.Tensor) -> torch.Tensor:
	h = self.conv_in(x) + x.repeat_interleave(repeats=self.repeat, dim=1)
	if torch.is_grad_enabled() and self.gradient_checkpointing:
	h = self._gradient_checkpointing_func(self.mid_block, h)
	else:
	h = self.mid_block(h)
	for up_block in self.up_blocks:
	if torch.is_grad_enabled() and self.gradient_checkpointing:
	h = self._gradient_checkpointing_func(up_block, h)

diffusers/src/diffusers/models/autoencoders/autoencoder_kl_hunyuanimage.py

Line 420 in 0f1abc4

_supports_gradient_checkpointing = False

Problem:
The encoder and decoder contain self._gradient_checkpointing_func(...) branches, but AutoencoderKLHunyuanImage._supports_gradient_checkpointing is False.

Impact:
Training or fine-tuning code cannot enable gradient checkpointing on the base VAE even though the implementation paths exist.

Reproduction:

from diffusers import AutoencoderKLHunyuanImage

vae = AutoencoderKLHunyuanImage(
 in_channels=3, out_channels=3, latent_channels=4, block_out_channels=(32,),
 layers_per_block=1, spatial_compression_ratio=1, sample_size=8,
)
vae.enable_gradient_checkpointing()

Relevant precedent:

diffusers/src/diffusers/models/autoencoders/autoencoder_kl_hunyuanimage_refiner.py

Line 602 in 0f1abc4

_supports_gradient_checkpointing = True

Suggested fix:

_supports_gradient_checkpointing = True

Issue 8: Test coverage is incomplete, including missing slow tests

Affected code:

diffusers/tests/pipelines/hunyuan_image_21/test_hunyuanimage.py

Lines 43 to 48 in 0f1abc4

	class HunyuanImagePipelineFastTests(
	PipelineTesterMixin,
	FirstBlockCacheTesterMixin,
	unittest.TestCase,
	):
	pipeline_class = HunyuanImagePipeline

Problem:
There is one fast pipeline test class for HunyuanImagePipeline. I found no model fast tests for HunyuanImageTransformer2DModel, AutoencoderKLHunyuanImage, or AutoencoderKLHunyuanImageRefiner, no fast tests for HunyuanImageRefinerPipeline, and no @slow tests for this family.

Impact:
The broken VAE encode tiling, refiner tiling, refiner pipeline paths, model patch-size behavior, and default scaling-factor failure are not covered.

Reproduction:

from pathlib import Path

files = list(Path("tests").rglob("*.py"))
hunyuan_files = [p for p in files if "hunyuan_image" in str(p).lower() or "hunyuanimage" in p.read_text(encoding="utf-8", errors="ignore")]
print([str(p) for p in hunyuan_files])
print("has_refiner_pipeline_test", any("HunyuanImageRefinerPipeline" in p.read_text(encoding="utf-8", errors="ignore") for p in hunyuan_files))
print("has_slow_test", any("@slow" in p.read_text(encoding="utf-8", errors="ignore") for p in hunyuan_files))

Relevant precedent:
Other families keep slow pipeline tests alongside fast tests, for example:

diffusers/tests/pipelines/wan/test_wan.py

Line 185 in 0f1abc4

@slow

Suggested fix:
Add focused model tests under tests/models/ for the transformer and both VAEs, add fast refiner pipeline tests with tiny components, and add at least one slow test for the base and refiner pretrained checkpoints. I attempted the existing fast Hunyuan test file locally with .venv, but collection failed before running tests because this environment's PyTorch build lacks torch._C._distributed_c10d.