hunyuan_video model/pipeline review

[hlky]

hunyuan_video model/pipeline review

Commit tested: 0f1abc4ae8b0eb2a3b40e82a310507281144c423

Review performed against the repository review rules.

Files/categories reviewed: public exports and lazy imports, pipeline runtime behavior, config/loading surfaces, dtype/device/offload-sensitive paths, model forward behavior, attention processor surfaces, fast/slow tests, and duplicate status.

Duplicate-search status: searched existing huggingface/diffusers Issues and PRs for hunyuan_video, affected class names, prompt_2, prompt_attention_mask, num_videos_per_prompt, and Framepack transformer failure modes. I did not find direct duplicates for the findings below. Related but not duplicate-adjacent batch issues exist for the older text-to-video pipeline, including #10453 and #10542.

Issue 1: prompt_2 is ignored by the CLIP encoder

Affected code:

diffusers/src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_video.py

Lines 327 to 336 in 0f1abc4

	if pooled_prompt_embeds is None:
	if prompt_2 is None:
	prompt_2 = prompt
	pooled_prompt_embeds = self._get_clip_prompt_embeds(
	prompt,
	num_videos_per_prompt,
	device=device,
	dtype=dtype,
	max_sequence_length=77,
	)

diffusers/src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_skyreels_image2video.py

Lines 356 to 365 in 0f1abc4

	if pooled_prompt_embeds is None:
	if prompt_2 is None:
	prompt_2 = prompt
	pooled_prompt_embeds = self._get_clip_prompt_embeds(
	prompt,
	num_videos_per_prompt,
	device=device,
	dtype=dtype,
	max_sequence_length=77,
	)

diffusers/src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_video_framepack.py

Lines 433 to 442 in 0f1abc4

	if pooled_prompt_embeds is None:
	if prompt_2 is None:
	prompt_2 = prompt
	pooled_prompt_embeds = self._get_clip_prompt_embeds(
	prompt,
	num_videos_per_prompt,
	device=device,
	dtype=dtype,
	max_sequence_length=77,
	)

diffusers/src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_video_image2video.py

Lines 483 to 492 in 0f1abc4

	if pooled_prompt_embeds is None:
	if prompt_2 is None:
	prompt_2 = prompt
	pooled_prompt_embeds = self._get_clip_prompt_embeds(
	prompt,
	num_videos_per_prompt,
	device=device,
	dtype=dtype,
	max_sequence_length=77,
	)

Problem:
All four pipelines normalize prompt_2 from prompt, but then pass prompt into _get_clip_prompt_embeds(...). As a result, user-provided prompt_2 and negative_prompt_2 never affect the CLIP branch.

Impact:
The public API exposes a second prompt input that silently does nothing. Users cannot independently condition the Llama and CLIP encoders, and negative CLIP prompting is also misapplied.

Reproduction:

import torch
from diffusers import HunyuanVideoPipeline

class Probe(HunyuanVideoPipeline):
 def __init__(self):
 pass

 def _get_llama_prompt_embeds(self, prompt, *args, **kwargs):
 return torch.zeros(1, 1, 1), torch.ones(1, 1)

 def _get_clip_prompt_embeds(self, prompt, *args, **kwargs):
 self.clip_prompt_seen = prompt
 return torch.zeros(1, 1)

pipe = Probe()
pipe.encode_prompt(
 prompt="main prompt",
 prompt_2="clip prompt",
 prompt_template={"template": "{}", "crop_start": 0},
)
print(pipe.clip_prompt_seen)
# main prompt

Relevant precedent:
The same pipelines already intend this split with if prompt_2 is None: prompt_2 = prompt; the bug is the wrong argument at the CLIP callsite.

Suggested fix:

pooled_prompt_embeds = self._get_clip_prompt_embeds(
 prompt_2,
 num_videos_per_prompt,
 device=device,
 dtype=dtype,
 max_sequence_length=77,
)

Apply this in the source pipeline and copied variants, then run the repository copy fixer for copied code.

Issue 2: Precomputed prompt embeds crash without attention masks

Affected code:

diffusers/src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_video.py

Lines 340 to 386 in 0f1abc4

	def check_inputs(
	self,
	prompt,
	prompt_2,
	height,
	width,
	prompt_embeds=None,
	callback_on_step_end_tensor_inputs=None,
	prompt_template=None,
	):
	if height % 16 != 0 or width % 16 != 0:
	raise ValueError(f"`height` and `width` have to be divisible by 16 but are {height} and {width}.")
	if callback_on_step_end_tensor_inputs is not None and not all(
	k in self._callback_tensor_inputs for k in callback_on_step_end_tensor_inputs
	):
	raise ValueError(
	f"`callback_on_step_end_tensor_inputs` has to be in {self._callback_tensor_inputs}, but found {[k for k in callback_on_step_end_tensor_inputs if k not in self._callback_tensor_inputs]}"
	)
	if prompt is not None and prompt_embeds is not None:
	raise ValueError(
	f"Cannot forward both `prompt`: {prompt} and `prompt_embeds`: {prompt_embeds}. Please make sure to"
	" only forward one of the two."
	)
	elif prompt_2 is not None and prompt_embeds is not None:
	raise ValueError(
	f"Cannot forward both `prompt_2`: {prompt_2} and `prompt_embeds`: {prompt_embeds}. Please make sure to"
	" only forward one of the two."
	)
	elif prompt is None and prompt_embeds is None:
	raise ValueError(
	"Provide either `prompt` or `prompt_embeds`. Cannot leave both `prompt` and `prompt_embeds` undefined."
	)
	elif prompt is not None and (not isinstance(prompt, str) and not isinstance(prompt, list)):
	raise ValueError(f"`prompt` has to be of type `str` or `list` but is {type(prompt)}")
	elif prompt_2 is not None and (not isinstance(prompt_2, str) and not isinstance(prompt_2, list)):
	raise ValueError(f"`prompt_2` has to be of type `str` or `list` but is {type(prompt_2)}")
	if prompt_template is not None:
	if not isinstance(prompt_template, dict):
	raise ValueError(f"`prompt_template` has to be of type `dict` but is {type(prompt_template)}")
	if "template" not in prompt_template:
	raise ValueError(
	f"`prompt_template` has to contain a key `template` but only found {prompt_template.keys()}"
	)

diffusers/src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_video.py

Lines 651 to 664 in 0f1abc4

	prompt_embeds, pooled_prompt_embeds, prompt_attention_mask = self.encode_prompt(
	prompt=prompt,
	prompt_2=prompt_2,
	prompt_template=prompt_template,
	num_videos_per_prompt=num_videos_per_prompt,
	prompt_embeds=prompt_embeds,
	pooled_prompt_embeds=pooled_prompt_embeds,
	prompt_attention_mask=prompt_attention_mask,
	device=device,
	max_sequence_length=max_sequence_length,
	)
	prompt_embeds = prompt_embeds.to(transformer_dtype)
	prompt_attention_mask = prompt_attention_mask.to(transformer_dtype)
	pooled_prompt_embeds = pooled_prompt_embeds.to(transformer_dtype)

diffusers/src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_skyreels_image2video.py

Lines 370 to 416 in 0f1abc4

	def check_inputs(
	self,
	prompt,
	prompt_2,
	height,
	width,
	prompt_embeds=None,
	callback_on_step_end_tensor_inputs=None,
	prompt_template=None,
	):
	if height % 16 != 0 or width % 16 != 0:
	raise ValueError(f"`height` and `width` have to be divisible by 16 but are {height} and {width}.")
	if callback_on_step_end_tensor_inputs is not None and not all(
	k in self._callback_tensor_inputs for k in callback_on_step_end_tensor_inputs
	):
	raise ValueError(
	f"`callback_on_step_end_tensor_inputs` has to be in {self._callback_tensor_inputs}, but found {[k for k in callback_on_step_end_tensor_inputs if k not in self._callback_tensor_inputs]}"
	)
	if prompt is not None and prompt_embeds is not None:
	raise ValueError(
	f"Cannot forward both `prompt`: {prompt} and `prompt_embeds`: {prompt_embeds}. Please make sure to"
	" only forward one of the two."
	)
	elif prompt_2 is not None and prompt_embeds is not None:
	raise ValueError(
	f"Cannot forward both `prompt_2`: {prompt_2} and `prompt_embeds`: {prompt_embeds}. Please make sure to"
	" only forward one of the two."
	)
	elif prompt is None and prompt_embeds is None:
	raise ValueError(
	"Provide either `prompt` or `prompt_embeds`. Cannot leave both `prompt` and `prompt_embeds` undefined."
	)
	elif prompt is not None and (not isinstance(prompt, str) and not isinstance(prompt, list)):
	raise ValueError(f"`prompt` has to be of type `str` or `list` but is {type(prompt)}")
	elif prompt_2 is not None and (not isinstance(prompt_2, str) and not isinstance(prompt_2, list)):
	raise ValueError(f"`prompt_2` has to be of type `str` or `list` but is {type(prompt_2)}")
	if prompt_template is not None:
	if not isinstance(prompt_template, dict):
	raise ValueError(f"`prompt_template` has to be of type `dict` but is {type(prompt_template)}")
	if "template" not in prompt_template:
	raise ValueError(
	f"`prompt_template` has to contain a key `template` but only found {prompt_template.keys()}"
	)

diffusers/src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_video_framepack.py

Lines 455 to 505 in 0f1abc4

	def check_inputs(
	self,
	prompt,
	prompt_2,
	height,
	width,
	prompt_embeds=None,
	callback_on_step_end_tensor_inputs=None,
	prompt_template=None,
	image=None,
	image_latents=None,
	last_image=None,
	last_image_latents=None,
	sampling_type=None,
	):
	if height % 16 != 0 or width % 16 != 0:
	raise ValueError(f"`height` and `width` have to be divisible by 16 but are {height} and {width}.")
	if callback_on_step_end_tensor_inputs is not None and not all(
	k in self._callback_tensor_inputs for k in callback_on_step_end_tensor_inputs
	):
	raise ValueError(
	f"`callback_on_step_end_tensor_inputs` has to be in {self._callback_tensor_inputs}, but found {[k for k in callback_on_step_end_tensor_inputs if k not in self._callback_tensor_inputs]}"
	)
	if prompt is not None and prompt_embeds is not None:
	raise ValueError(
	f"Cannot forward both `prompt`: {prompt} and `prompt_embeds`: {prompt_embeds}. Please make sure to"
	" only forward one of the two."
	)
	elif prompt_2 is not None and prompt_embeds is not None:
	raise ValueError(
	f"Cannot forward both `prompt_2`: {prompt_2} and `prompt_embeds`: {prompt_embeds}. Please make sure to"
	" only forward one of the two."
	)
	elif prompt is None and prompt_embeds is None:
	raise ValueError(
	"Provide either `prompt` or `prompt_embeds`. Cannot leave both `prompt` and `prompt_embeds` undefined."
	)
	elif prompt is not None and (not isinstance(prompt, str) and not isinstance(prompt, list)):
	raise ValueError(f"`prompt` has to be of type `str` or `list` but is {type(prompt)}")
	elif prompt_2 is not None and (not isinstance(prompt_2, str) and not isinstance(prompt_2, list)):
	raise ValueError(f"`prompt_2` has to be of type `str` or `list` but is {type(prompt_2)}")
	if prompt_template is not None:
	if not isinstance(prompt_template, dict):
	raise ValueError(f"`prompt_template` has to be of type `dict` but is {type(prompt_template)}")
	if "template" not in prompt_template:
	raise ValueError(
	f"`prompt_template` has to contain a key `template` but only found {prompt_template.keys()}"
	)

diffusers/src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_video_image2video.py

Lines 496 to 550 in 0f1abc4

	def check_inputs(
	self,
	prompt,
	prompt_2,
	height,
	width,
	prompt_embeds=None,
	callback_on_step_end_tensor_inputs=None,
	prompt_template=None,
	true_cfg_scale=1.0,
	guidance_scale=1.0,
	):
	if height % 16 != 0 or width % 16 != 0:
	raise ValueError(f"`height` and `width` have to be divisible by 16 but are {height} and {width}.")
	if callback_on_step_end_tensor_inputs is not None and not all(
	k in self._callback_tensor_inputs for k in callback_on_step_end_tensor_inputs
	):
	raise ValueError(
	f"`callback_on_step_end_tensor_inputs` has to be in {self._callback_tensor_inputs}, but found {[k for k in callback_on_step_end_tensor_inputs if k not in self._callback_tensor_inputs]}"
	)
	if prompt is not None and prompt_embeds is not None:
	raise ValueError(
	f"Cannot forward both `prompt`: {prompt} and `prompt_embeds`: {prompt_embeds}. Please make sure to"
	" only forward one of the two."
	)
	elif prompt_2 is not None and prompt_embeds is not None:
	raise ValueError(
	f"Cannot forward both `prompt_2`: {prompt_2} and `prompt_embeds`: {prompt_embeds}. Please make sure to"
	" only forward one of the two."
	)
	elif prompt is None and prompt_embeds is None:
	raise ValueError(
	"Provide either `prompt` or `prompt_embeds`. Cannot leave both `prompt` and `prompt_embeds` undefined."
	)
	elif prompt is not None and (not isinstance(prompt, str) and not isinstance(prompt, list)):
	raise ValueError(f"`prompt` has to be of type `str` or `list` but is {type(prompt)}")
	elif prompt_2 is not None and (not isinstance(prompt_2, str) and not isinstance(prompt_2, list)):
	raise ValueError(f"`prompt_2` has to be of type `str` or `list` but is {type(prompt_2)}")
	if prompt_template is not None:
	if not isinstance(prompt_template, dict):
	raise ValueError(f"`prompt_template` has to be of type `dict` but is {type(prompt_template)}")
	if "template" not in prompt_template:
	raise ValueError(
	f"`prompt_template` has to contain a key `template` but only found {prompt_template.keys()}"
	)
	if true_cfg_scale > 1.0 and guidance_scale > 1.0:
	logger.warning(
	"Both `true_cfg_scale` and `guidance_scale` are greater than 1.0. This will result in both "
	"classifier-free guidance and embedded-guidance to be applied. This is not recommended "
	"as it may lead to higher memory usage, slower inference and potentially worse results."
	)

Problem:
The pipelines allow prompt_embeds to be supplied directly, but check_inputs does not require prompt_attention_mask. encode_prompt returns None for the mask, and __call__ later executes prompt_attention_mask.to(...).

Impact:
The documented embed-based path fails with an opaque AttributeError instead of a validation error. The same gap exists for negative prompt embeddings and pooled embeddings.

Reproduction:

import torch
from diffusers import HunyuanVideoPipeline

pipe = object.__new__(HunyuanVideoPipeline)
prompt_embeds = torch.zeros(1, 2, 3)
pooled_prompt_embeds = torch.zeros(1, 4)

try:
 HunyuanVideoPipeline.check_inputs(
 pipe,
 prompt=None,
 prompt_2=None,
 height=16,
 width=16,
 prompt_embeds=prompt_embeds,
 callback_on_step_end_tensor_inputs=None,
 prompt_template={"template": "{}"},
 )
 _, _, prompt_attention_mask = HunyuanVideoPipeline.encode_prompt(
 pipe,
 prompt=None,
 prompt_embeds=prompt_embeds,
 pooled_prompt_embeds=pooled_prompt_embeds,
 prompt_attention_mask=None,
 )
 prompt_attention_mask.to(torch.float32)
except Exception as e:
 print(type(e).__name__, e)
# AttributeError 'NoneType' object has no attribute 'to'

Relevant precedent:

diffusers/src/diffusers/pipelines/ltx/pipeline_ltx.py

Lines 399 to 403 in 0f1abc4

	if prompt_embeds is not None and prompt_attention_mask is None:
	raise ValueError("Must provide `prompt_attention_mask` when specifying `prompt_embeds`.")
	if negative_prompt_embeds is not None and negative_prompt_attention_mask is None:
	raise ValueError("Must provide `negative_prompt_attention_mask` when specifying `negative_prompt_embeds`.")

Suggested fix:

if prompt_embeds is not None and prompt_attention_mask is None:
 raise ValueError("Must provide `prompt_attention_mask` when specifying `prompt_embeds`.")
if prompt_embeds is not None and pooled_prompt_embeds is None:
 raise ValueError("Must provide `pooled_prompt_embeds` when specifying `prompt_embeds`.")
if negative_prompt_embeds is not None and negative_prompt_attention_mask is None:
 raise ValueError("Must provide `negative_prompt_attention_mask` when specifying `negative_prompt_embeds`.")
if negative_prompt_embeds is not None and negative_pooled_prompt_embeds is None:
 raise ValueError("Must provide `negative_pooled_prompt_embeds` when specifying `negative_prompt_embeds`.")

Issue 3: Image-conditioned pipelines break num_videos_per_prompt > 1

Affected code:

diffusers/src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_video_image2video.py

Lines 552 to 599 in 0f1abc4

	def prepare_latents(
	self,
	image: torch.Tensor,
	batch_size: int,
	num_channels_latents: int = 32,
	height: int = 720,
	width: int = 1280,
	num_frames: int = 129,
	dtype: torch.dtype | None = None,
	device: torch.device | None = None,
	generator: torch.Generator | list[torch.Generator] | None = None,
	latents: torch.Tensor | None = None,
	image_condition_type: str = "latent_concat",
	) -> torch.Tensor:
	if isinstance(generator, list) and len(generator) != batch_size:
	raise ValueError(
	f"You have passed a list of generators of length {len(generator)}, but requested an effective batch"
	f" size of {batch_size}. Make sure the batch size matches the length of the generators."
	)
	num_latent_frames = (num_frames - 1) // self.vae_scale_factor_temporal + 1
	latent_height, latent_width = height // self.vae_scale_factor_spatial, width // self.vae_scale_factor_spatial
	shape = (batch_size, num_channels_latents, num_latent_frames, latent_height, latent_width)
	image = image.unsqueeze(2) # [B, C, 1, H, W]
	if isinstance(generator, list):
	image_latents = [
	retrieve_latents(self.vae.encode(image[i].unsqueeze(0)), generator[i], "argmax")
	for i in range(batch_size)
	]
	else:
	image_latents = [retrieve_latents(self.vae.encode(img.unsqueeze(0)), generator, "argmax") for img in image]
	image_latents = torch.cat(image_latents, dim=0).to(dtype) * self.vae_scaling_factor
	image_latents = image_latents.repeat(1, 1, num_latent_frames, 1, 1)
	if latents is None:
	latents = randn_tensor(shape, generator=generator, device=device, dtype=dtype)
	else:
	latents = latents.to(device=device, dtype=dtype)
	t = torch.tensor([0.999]).to(device=device)
	latents = latents * t + image_latents * (1 - t)
	if image_condition_type == "token_replace":
	image_latents = image_latents[:, :, :1]
	return latents, image_latents

diffusers/src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_skyreels_image2video.py

Lines 417 to 459 in 0f1abc4

	def prepare_latents(
	self,
	image: torch.Tensor,
	batch_size: int,
	num_channels_latents: int = 32,
	height: int = 544,
	width: int = 960,
	num_frames: int = 97,
	dtype: torch.dtype | None = None,
	device: torch.device | None = None,
	generator: torch.Generator | list[torch.Generator] | None = None,
	latents: torch.Tensor | None = None,
	) -> torch.Tensor:
	if isinstance(generator, list) and len(generator) != batch_size:
	raise ValueError(
	f"You have passed a list of generators of length {len(generator)}, but requested an effective batch"
	f" size of {batch_size}. Make sure the batch size matches the length of the generators."
	)
	image = image.unsqueeze(2) # [B, C, 1, H, W]
	if isinstance(generator, list):
	image_latents = [
	retrieve_latents(self.vae.encode(image[i].unsqueeze(0)), generator[i]) for i in range(batch_size)
	]
	else:
	image_latents = [retrieve_latents(self.vae.encode(img.unsqueeze(0)), generator) for img in image]
	image_latents = torch.cat(image_latents, dim=0).to(dtype) * self.vae_scaling_factor
	num_latent_frames = (num_frames - 1) // self.vae_scale_factor_temporal + 1
	latent_height, latent_width = height // self.vae_scale_factor_spatial, width // self.vae_scale_factor_spatial
	shape = (batch_size, num_channels_latents, num_latent_frames, latent_height, latent_width)
	padding_shape = (batch_size, num_channels_latents, num_latent_frames - 1, latent_height, latent_width)
	latents_padding = torch.zeros(padding_shape, dtype=dtype, device=device)
	image_latents = torch.cat([image_latents, latents_padding], dim=2)
	if latents is None:
	latents = randn_tensor(shape, generator=generator, dtype=dtype, device=device)
	else:
	latents = latents.to(dtype=dtype, device=device)
	return latents, image_latents

diffusers/src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_video_framepack.py

Lines 551 to 564 in 0f1abc4

	def prepare_image_latents(
	self,
	image: torch.Tensor,
	dtype: torch.dtype | None = None,
	device: torch.device | None = None,
	generator: torch.Generator | list[torch.Generator] | None = None,
	latents: torch.Tensor | None = None,
	) -> torch.Tensor:
	device = device or self._execution_device
	if latents is None:
	image = image.unsqueeze(2).to(device=device, dtype=self.vae.dtype)
	latents = self.vae.encode(image).latent_dist.sample(generator=generator)
	latents = latents * self.vae.config.scaling_factor
	return latents.to(device=device, dtype=dtype)

diffusers/src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_video_framepack.py

Lines 908 to 961 in 0f1abc4

	guidance = torch.tensor([guidance_scale] * batch_size, dtype=transformer_dtype, device=device) * 1000.0
	# 7. Denoising loop
	for k in range(num_latent_sections):
	if sampling_type == FramepackSamplingType.INVERTED_ANTI_DRIFTING:
	latent_paddings = list(reversed(range(num_latent_sections)))
	if num_latent_sections > 4:
	latent_paddings = [3] + [2] * (num_latent_sections - 3) + [1, 0]
	is_first_section = k == 0
	is_last_section = k == num_latent_sections - 1
	latent_padding_size = latent_paddings[k] * latent_window_size
	indices = torch.arange(0, sum([1, latent_padding_size, latent_window_size, *history_sizes]))
	(
	indices_prefix,
	indices_padding,
	indices_latents,
	indices_latents_history_1x,
	indices_latents_history_2x,
	indices_latents_history_4x,
	) = indices.split([1, latent_padding_size, latent_window_size, *history_sizes], dim=0)
	# Inverted anti-drifting sampling: Figure 2(c) in the paper
	indices_clean_latents = torch.cat([indices_prefix, indices_latents_history_1x], dim=0)
	latents_prefix = image_latents
	latents_history_1x, latents_history_2x, latents_history_4x = history_latents[
	:, :, : sum(history_sizes)
	].split(history_sizes, dim=2)
	if last_image is not None and is_first_section:
	latents_history_1x = last_image_latents
	latents_clean = torch.cat([latents_prefix, latents_history_1x], dim=2)
	elif sampling_type == FramepackSamplingType.VANILLA:
	indices = torch.arange(0, sum([1, *history_sizes, latent_window_size]))
	(
	indices_prefix,
	indices_latents_history_4x,
	indices_latents_history_2x,
	indices_latents_history_1x,
	indices_latents,
	) = indices.split([1, *history_sizes, latent_window_size], dim=0)
	indices_clean_latents = torch.cat([indices_prefix, indices_latents_history_1x], dim=0)
	latents_prefix = image_latents
	latents_history_4x, latents_history_2x, latents_history_1x = history_latents[
	:, :, -sum(history_sizes) :
	].split(history_sizes, dim=2)
	latents_clean = torch.cat([latents_prefix, latents_history_1x], dim=2)
	else:
	assert False
	latents = self.prepare_latents(

Problem:
The image-conditioned pipelines compute an effective batch size using batch_size * num_videos_per_prompt, but image latents are encoded once and not expanded to the effective batch. The I2V pipeline also does not repeat prompt embeddings in its prompt encoder helpers. Framepack duplicates text embeddings but still prepares guidance, history latents, and denoising latents using the original batch size.

Impact:
num_videos_per_prompt > 1 can produce tensor batch mismatches or internally inconsistent conditioning, especially when a single image should generate multiple videos.

Reproduction:

import torch
from diffusers import HunyuanVideoImageToVideoPipeline

class Dist:
 def __init__(self, latents):
 self._latents = latents
 def mode(self):
 return self._latents

class EncOut:
 def __init__(self, latents):
 self.latent_dist = Dist(latents)

class VAE:
 def encode(self, x):
 return EncOut(torch.zeros(x.shape[0], 4, 1, 2, 2))

pipe = object.__new__(HunyuanVideoImageToVideoPipeline)
pipe.vae = VAE()
pipe.vae_scaling_factor = 1.0
pipe.vae_scale_factor_temporal = 4
pipe.vae_scale_factor_spatial = 8

latents, image_latents = HunyuanVideoImageToVideoPipeline.prepare_latents(
 pipe,
 image=torch.zeros(1, 3, 16, 16),
 batch_size=2,
 num_channels_latents=4,
 height=16,
 width=16,
 num_frames=9,
 dtype=torch.float32,
 device=torch.device("cpu"),
 generator=None,
 latents=None,
 image_condition_type="latent_concat",
)

print(latents.shape[0], image_latents.shape[0])
# 2 1

Relevant precedent:

diffusers/src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_video.py

Lines 256 to 261 in 0f1abc4

	# duplicate text embeddings for each generation per prompt, using mps friendly method
	_, seq_len, _ = prompt_embeds.shape
	prompt_embeds = prompt_embeds.repeat(1, num_videos_per_prompt, 1)
	prompt_embeds = prompt_embeds.view(batch_size * num_videos_per_prompt, seq_len, -1)
	prompt_attention_mask = prompt_attention_mask.repeat(1, num_videos_per_prompt)
	prompt_attention_mask = prompt_attention_mask.view(batch_size * num_videos_per_prompt, seq_len)

diffusers/src/diffusers/pipelines/hunyuan_video/pipeline_hunyuan_video.py

Lines 694 to 704 in 0f1abc4

	latents = self.prepare_latents(
	batch_size * num_videos_per_prompt,
	num_channels_latents,
	height,
	width,
	num_frames,
	torch.float32,
	device,
	generator,
	latents,
	)

Suggested fix:

def _repeat_to_effective_batch(tensor, effective_batch_size):
 if tensor.shape[0] == effective_batch_size:
 return tensor
 if effective_batch_size % tensor.shape[0] != 0:
 raise ValueError("Conditioning batch size must divide the effective generation batch size.")
 return tensor.repeat_interleave(effective_batch_size // tensor.shape[0], dim=0)

Use this for image latents and image embeddings. Also repeat I2V prompt embeddings like the text-to-video pipeline, and make Framepack use effective_batch_size = batch_size * num_videos_per_prompt consistently for guidance, history, and latent preparation.

Issue 4: Framepack transformer optional config paths crash

Affected code:

diffusers/src/diffusers/models/transformers/transformer_hunyuan_video_framepack.py

Lines 124 to 166 in 0f1abc4

	@register_to_config
	def __init__(
	self,
	in_channels: int = 16,
	out_channels: int = 16,
	num_attention_heads: int = 24,
	attention_head_dim: int = 128,
	num_layers: int = 20,
	num_single_layers: int = 40,
	num_refiner_layers: int = 2,
	mlp_ratio: float = 4.0,
	patch_size: int = 2,
	patch_size_t: int = 1,
	qk_norm: str = "rms_norm",
	guidance_embeds: bool = True,
	text_embed_dim: int = 4096,
	pooled_projection_dim: int = 768,
	rope_theta: float = 256.0,
	rope_axes_dim: tuple[int, ...] = (16, 56, 56),
	image_condition_type: str | None = None,
	has_image_proj: int = False,
	image_proj_dim: int = 1152,
	has_clean_x_embedder: int = False,
	) -> None:
	super().__init__()
	inner_dim = num_attention_heads * attention_head_dim
	out_channels = out_channels or in_channels
	# 1. Latent and condition embedders
	self.x_embedder = HunyuanVideoPatchEmbed((patch_size_t, patch_size, patch_size), in_channels, inner_dim)
	# Framepack history projection embedder
	self.clean_x_embedder = None
	if has_clean_x_embedder:
	self.clean_x_embedder = HunyuanVideoHistoryPatchEmbed(in_channels, inner_dim)
	self.context_embedder = HunyuanVideoTokenRefiner(
	text_embed_dim, num_attention_heads, attention_head_dim, num_layers=num_refiner_layers
	)
	# Framepack image-conditioning embedder
	self.image_projection = FramepackClipVisionProjection(image_proj_dim, inner_dim) if has_image_proj else None

diffusers/src/diffusers/models/transformers/transformer_hunyuan_video_framepack.py

Lines 236 to 269 in 0f1abc4

	latents_clean, latents_history_2x, latents_history_4x = self.clean_x_embedder(
	latents_clean, latents_history_2x, latents_history_4x
	)
	if latents_clean is not None and indices_latents_clean is not None:
	image_rotary_emb_clean = self.rope(
	frame_indices=indices_latents_clean, height=height, width=width, device=hidden_states.device
	)
	if latents_history_2x is not None and indices_latents_history_2x is not None:
	image_rotary_emb_history_2x = self.rope(
	frame_indices=indices_latents_history_2x, height=height, width=width, device=hidden_states.device
	)
	if latents_history_4x is not None and indices_latents_history_4x is not None:
	image_rotary_emb_history_4x = self.rope(
	frame_indices=indices_latents_history_4x, height=height, width=width, device=hidden_states.device
	)
	hidden_states, image_rotary_emb = self._pack_history_states(
	hidden_states,
	latents_clean,
	latents_history_2x,
	latents_history_4x,
	image_rotary_emb,
	image_rotary_emb_clean,
	image_rotary_emb_history_2x,
	image_rotary_emb_history_4x,
	post_patch_height,
	post_patch_width,
	)
	temb, _ = self.time_text_embed(timestep, pooled_projections, guidance)
	encoder_hidden_states = self.context_embedder(encoder_hidden_states, timestep, encoder_attention_mask)
	encoder_hidden_states_image = self.image_projection(image_embeds)

Problem:
HunyuanVideoFramepackTransformer3DModel defaults has_image_proj=False and has_clean_x_embedder=False, but forward unconditionally calls self.clean_x_embedder(...) and self.image_projection(...). It also passes optional rotary embedding locals that are only defined inside conditional branches.

Impact:
Several serialized config combinations that the class advertises as valid fail at runtime. This weakens config backwards compatibility and makes minimal/tiny test fixtures harder to construct.

Reproduction:

import torch
from diffusers import HunyuanVideoFramepackTransformer3DModel

model = HunyuanVideoFramepackTransformer3DModel(
 in_channels=4,
 out_channels=4,
 num_attention_heads=2,
 attention_head_dim=4,
 num_layers=1,
 num_single_layers=1,
 num_refiner_layers=1,
 patch_size=2,
 patch_size_t=1,
 guidance_embeds=True,
 text_embed_dim=8,
 pooled_projection_dim=6,
 rope_axes_dim=(2, 2, 4),
 has_image_proj=True,
 image_proj_dim=8,
 has_clean_x_embedder=False,
).eval()

try:
 with torch.no_grad():
 model(
 hidden_states=torch.randn(1, 4, 1, 4, 4),
 timestep=torch.tensor([1]),
 encoder_hidden_states=torch.randn(1, 3, 8),
 encoder_attention_mask=torch.ones(1, 3),
 pooled_projections=torch.randn(1, 6),
 image_embeds=torch.randn(1, 2, 8),
 indices_latents=torch.arange(1),
 guidance=torch.tensor([1.0]),
 )
except Exception as e:
 print(type(e).__name__, e)
# TypeError 'NoneType' object is not callable

Relevant precedent:
The non-Framepack Hunyuan transformer keeps optional projections/configured modules aligned with its forward path rather than exposing defaults that immediately fail.

Suggested fix:

image_rotary_emb_clean = None
image_rotary_emb_history_2x = None
image_rotary_emb_history_4x = None

if self.clean_x_embedder is not None:
 latents_clean, latents_history_2x, latents_history_4x = self.clean_x_embedder(
 latents_clean, latents_history_2x, latents_history_4x
 )

if self.image_projection is not None and image_embeds is not None:
 encoder_hidden_states_image = self.image_projection(image_embeds)
 attention_mask_image = encoder_attention_mask.new_ones(
 (batch_size, encoder_hidden_states_image.shape[1])
 )
 encoder_hidden_states = torch.cat([encoder_hidden_states_image, encoder_hidden_states], dim=1)
 encoder_attention_mask = torch.cat([attention_mask_image, encoder_attention_mask], dim=1)

If released Framepack checkpoints always require these modules, the safer alternative is to change the config defaults and validate required inputs early with clear errors.

Issue 5: Slow tests are missing for the Hunyuan Video family

Affected code:

diffusers/tests/pipelines/hunyuan_video/test_hunyuan_video.py

Lines 348 to 357 in 0f1abc4

	@unittest.skip(
	"A very small vocab size is used for fast tests. So, Any kind of prompt other than the empty default used in other tests will lead to a embedding lookup error. This test uses a long prompt that causes the error."
	)
	def test_inference_batch_consistent(self):
	pass
	@unittest.skip(
	"A very small vocab size is used for fast tests. So, Any kind of prompt other than the empty default used in other tests will lead to a embedding lookup error. This test uses a long prompt that causes the error."
	)
	def test_inference_batch_single_identical(self):

diffusers/tests/pipelines/hunyuan_video/test_hunyuan_image2video.py

Lines 374 to 389 in 0f1abc4

	@unittest.skip(
	"A very small vocab size is used for fast tests. So, Any kind of prompt other than the empty default used in other tests will lead to a embedding lookup error. This test uses a long prompt that causes the error."
	)
	def test_inference_batch_consistent(self):
	pass
	@unittest.skip(
	"A very small vocab size is used for fast tests. So, Any kind of prompt other than the empty default used in other tests will lead to a embedding lookup error. This test uses a long prompt that causes the error."
	)
	def test_inference_batch_single_identical(self):
	pass
	@unittest.skip(
	"Encode prompt currently does not work in isolation because of requiring image embeddings from image processor. The test does not handle this case, or we need to rewrite encode_prompt."
	)
	def test_encode_prompt_works_in_isolation(self):

diffusers/tests/pipelines/hunyuan_video/test_hunyuan_skyreels_image2video.py

Lines 335 to 344 in 0f1abc4

	@unittest.skip(
	"A very small vocab size is used for fast tests. So, Any kind of prompt other than the empty default used in other tests will lead to a embedding lookup error. This test uses a long prompt that causes the error."
	)
	def test_inference_batch_consistent(self):
	pass
	@unittest.skip(
	"A very small vocab size is used for fast tests. So, Any kind of prompt other than the empty default used in other tests will lead to a embedding lookup error. This test uses a long prompt that causes the error."
	)
	def test_inference_batch_single_identical(self):

diffusers/tests/pipelines/hunyuan_video/test_hunyuan_video_framepack.py

Lines 394 to 403 in 0f1abc4

	@unittest.skip(
	"A very small vocab size is used for fast tests. So, Any kind of prompt other than the empty default used in other tests will lead to a embedding lookup error. This test uses a long prompt that causes the error."
	)
	def test_inference_batch_consistent(self):
	pass
	@unittest.skip(
	"A very small vocab size is used for fast tests. So, Any kind of prompt other than the empty default used in other tests will lead to a embedding lookup error. This test uses a long prompt that causes the error."
	)
	def test_inference_batch_single_identical(self):

Problem:
Fast tests exist for the Hunyuan Video pipelines and models, but I found no @slow pipeline tests under tests/pipelines/hunyuan_video. Several batch consistency tests are explicitly skipped, including for the image-conditioned and Framepack pipelines.

Impact:
Real-checkpoint smoke coverage is missing for this family, and the skipped batch tests leave the num_videos_per_prompt regressions above unguarded.

Reproduction:

from pathlib import Path

files = sorted(Path("tests/pipelines/hunyuan_video").glob("test_*.py"))
print({str(path): ("@slow" in path.read_text()) for path in files})
# All entries are False.

Relevant precedent:
Other major pipeline families include slow smoke tests for real checkpoint loading, basic inference, and scheduler/device behavior.

Suggested fix:
Add slow smoke tests for HunyuanVideoPipeline, HunyuanVideoImageToVideoPipeline, HunyuanSkyreelsImageToVideoPipeline, and HunyuanVideoFramepackPipeline. Add focused fast coverage for prompt_2, precomputed prompt embeds with masks, and num_videos_per_prompt=2; then unskip the batch consistency tests once the pipeline behavior is fixed.