lumina2 model/pipeline review

[hlky]

lumina2 model/pipeline review

Commit tested: 0f1abc4ae8b0eb2a3b40e82a310507281144c423

Review performed against the repository review rules.

Reviewed target pipeline/model/init files, top-level lazy exports, dummy exports, fast/model/single-file/LoRA tests, docs, and DreamBooth example coverage. Duplicate search was run with gh search issues/prs for lumina2, affected class/function names, and each specific failure mode. Only the scheduler image_seq_len item had an existing duplicate.

Targeted pytest command using .venv was attempted, but collection failed in this local torch build because torch._C._distributed_c10d is missing.

Issue 1: Deprecated alias is exported but cannot be constructed

Affected code:

diffusers/src/diffusers/pipelines/lumina2/pipeline_lumina2.py

Lines 801 to 818 in 0f1abc4

	class Lumina2Text2ImgPipeline(Lumina2Pipeline):
	def __init__(
	self,
	transformer: Lumina2Transformer2DModel,
	scheduler: FlowMatchEulerDiscreteScheduler,
	vae: AutoencoderKL,
	text_encoder: Gemma2PreTrainedModel,
	tokenizer: GemmaTokenizer | GemmaTokenizerFast,
	):
	deprecation_message = "`Lumina2Text2ImgPipeline` has been renamed to `Lumina2Pipeline` and will be removed in a future version. Please use `Lumina2Pipeline` instead."
	deprecate("diffusers.pipelines.lumina2.pipeline_lumina2.Lumina2Text2ImgPipeline", "0.34", deprecation_message)
	super().__init__(
	transformer=transformer,
	scheduler=scheduler,
	vae=vae,
	text_encoder=text_encoder,
	tokenizer=tokenizer,
	)

diffusers/src/diffusers/__init__.py

Lines 622 to 623 in 0f1abc4

	"Lumina2Pipeline",
	"Lumina2Text2ImgPipeline",

Problem:
Lumina2Text2ImgPipeline remains publicly exported, but its constructor calls deprecate(..., "0.34", ...). The current package version is 0.38.0.dev0, so deprecate raises a ValueError instead of warning.

Impact:
Users can import the backwards-compatible alias, but any construction or config load path that instantiates it fails immediately.

Reproduction:

from diffusers import Lumina2Text2ImgPipeline

Lumina2Text2ImgPipeline(
 transformer=None,
 scheduler=None,
 vae=None,
 text_encoder=None,
 tokenizer=None,
)

Relevant precedent:
Related rename PR, but not a duplicate for the current failure: #10827

Suggested fix:

# If keeping the alias:
deprecate(
 "diffusers.pipelines.lumina2.pipeline_lumina2.Lumina2Text2ImgPipeline",
 "1.0.0",
 deprecation_message,
)

# Or remove the alias from pipeline_lumina2.py, lazy exports, top-level exports, and dummy objects.

Issue 2: Precomputed negative prompt embeds are not repeated for num_images_per_prompt

Affected code:

diffusers/src/diffusers/pipelines/lumina2/pipeline_lumina2.py

Lines 297 to 339 in 0f1abc4

	batch_size, seq_len, _ = prompt_embeds.shape
	# duplicate text embeddings and attention mask for each generation per prompt, using mps friendly method
	prompt_embeds = prompt_embeds.repeat(1, num_images_per_prompt, 1)
	prompt_embeds = prompt_embeds.view(batch_size * num_images_per_prompt, seq_len, -1)
	prompt_attention_mask = prompt_attention_mask.repeat(num_images_per_prompt, 1)
	prompt_attention_mask = prompt_attention_mask.view(batch_size * num_images_per_prompt, -1)
	# Get negative embeddings for classifier free guidance
	if do_classifier_free_guidance and negative_prompt_embeds is None:
	negative_prompt = negative_prompt if negative_prompt is not None else ""
	# Normalize str to list
	negative_prompt = batch_size * [negative_prompt] if isinstance(negative_prompt, str) else negative_prompt
	if prompt is not None and type(prompt) is not type(negative_prompt):
	raise TypeError(
	f"`negative_prompt` should be the same type to `prompt`, but got {type(negative_prompt)} !="
	f" {type(prompt)}."
	)
	elif isinstance(negative_prompt, str):
	negative_prompt = [negative_prompt]
	elif batch_size != len(negative_prompt):
	raise ValueError(
	f"`negative_prompt`: {negative_prompt} has batch size {len(negative_prompt)}, but `prompt`:"
	f" {prompt} has batch size {batch_size}. Please make sure that passed `negative_prompt` matches"
	" the batch size of `prompt`."
	)
	negative_prompt_embeds, negative_prompt_attention_mask = self._get_gemma_prompt_embeds(
	prompt=negative_prompt,
	device=device,
	max_sequence_length=max_sequence_length,
	)
	batch_size, seq_len, _ = negative_prompt_embeds.shape
	# duplicate text embeddings and attention mask for each generation per prompt, using mps friendly method
	negative_prompt_embeds = negative_prompt_embeds.repeat(1, num_images_per_prompt, 1)
	negative_prompt_embeds = negative_prompt_embeds.view(batch_size * num_images_per_prompt, seq_len, -1)
	negative_prompt_attention_mask = negative_prompt_attention_mask.repeat(num_images_per_prompt, 1)
	negative_prompt_attention_mask = negative_prompt_attention_mask.view(
	batch_size * num_images_per_prompt, -1
	)
	return prompt_embeds, prompt_attention_mask, negative_prompt_embeds, negative_prompt_attention_mask

Problem:
encode_prompt repeats prompt_embeds and prompt_attention_mask unconditionally, but repeats negative_prompt_embeds and negative_prompt_attention_mask only when the pipeline encoded them itself. If the caller supplies precomputed negative embeddings and num_images_per_prompt > 1, the positive and negative batches diverge.

Impact:
Classifier-free guidance can broadcast incorrectly for batch size 1, or fail with a shape mismatch for larger batches.

Reproduction:

import torch
from diffusers import Lumina2Pipeline

pipe = Lumina2Pipeline(transformer=None, scheduler=None, vae=None, text_encoder=None, tokenizer=None)
pe, pm, ne, nm = pipe.encode_prompt(
 prompt=None,
 do_classifier_free_guidance=True,
 num_images_per_prompt=2,
 prompt_embeds=torch.randn(2, 3, 4),
 negative_prompt_embeds=torch.randn(2, 3, 4),
 prompt_attention_mask=torch.ones(2, 3, dtype=torch.bool),
 negative_prompt_attention_mask=torch.ones(2, 3, dtype=torch.bool),
)
print(pe.shape, pm.shape, ne.shape, nm.shape)
# positive batch is 4, negative batch is still 2

Relevant precedent:
LuminaPipeline repeats generated negative embeddings and masks with the positive batch.

diffusers/src/diffusers/pipelines/lumina/pipeline_lumina.py

Lines 360 to 365 in 0f1abc4

	negative_prompt_embeds = negative_prompt_embeds.to(dtype=negative_dtype, device=device)
	# duplicate text embeddings and attention mask for each generation per prompt, using mps friendly method
	negative_prompt_embeds = negative_prompt_embeds.repeat(1, num_images_per_prompt, 1)
	negative_prompt_embeds = negative_prompt_embeds.view(batch_size * num_images_per_prompt, seq_len, -1)
	negative_prompt_attention_mask = negative_prompt_attention_mask.repeat(num_images_per_prompt, 1)
	negative_prompt_attention_mask = negative_prompt_attention_mask.view(

Suggested fix:

if do_classifier_free_guidance:
 if negative_prompt_embeds is None:
 negative_prompt_embeds, negative_prompt_attention_mask = self._get_gemma_prompt_embeds(...)

 negative_batch_size, neg_seq_len, _ = negative_prompt_embeds.shape
 negative_prompt_embeds = negative_prompt_embeds.repeat(1, num_images_per_prompt, 1)
 negative_prompt_embeds = negative_prompt_embeds.view(
 negative_batch_size * num_images_per_prompt, neg_seq_len, -1
 )
 negative_prompt_attention_mask = negative_prompt_attention_mask.repeat(num_images_per_prompt, 1)
 negative_prompt_attention_mask = negative_prompt_attention_mask.view(
 negative_batch_size * num_images_per_prompt, -1
 )

Issue 3: Precomputed prompt embeds are not cast to transformer dtype

Affected code:

diffusers/src/diffusers/pipelines/lumina2/pipeline_lumina2.py

Lines 290 to 302 in 0f1abc4

	if prompt_embeds is None:
	prompt_embeds, prompt_attention_mask = self._get_gemma_prompt_embeds(
	prompt=prompt,
	device=device,
	max_sequence_length=max_sequence_length,
	)
	batch_size, seq_len, _ = prompt_embeds.shape
	# duplicate text embeddings and attention mask for each generation per prompt, using mps friendly method
	prompt_embeds = prompt_embeds.repeat(1, num_images_per_prompt, 1)
	prompt_embeds = prompt_embeds.view(batch_size * num_images_per_prompt, seq_len, -1)
	prompt_attention_mask = prompt_attention_mask.repeat(num_images_per_prompt, 1)
	prompt_attention_mask = prompt_attention_mask.view(batch_size * num_images_per_prompt, -1)

diffusers/src/diffusers/pipelines/lumina2/pipeline_lumina2.py

Lines 684 to 692 in 0f1abc4

	# 4. Prepare latents.
	latent_channels = self.transformer.config.in_channels
	latents = self.prepare_latents(
	batch_size * num_images_per_prompt,
	latent_channels,
	height,
	width,
	prompt_embeds.dtype,
	device,

Problem:
When prompt_embeds are supplied directly, encode_prompt does not cast them to the transformer dtype. __call__ then creates latents with prompt_embeds.dtype, so float32 prompt embeds plus a bf16 transformer produce float32 latents fed into bf16 linear layers.

Impact:
Common precomputed-embedding workflows fail for bf16/quantized transformer usage.

Reproduction:

import torch
from diffusers import FlowMatchEulerDiscreteScheduler, Lumina2Pipeline, Lumina2Transformer2DModel

transformer = Lumina2Transformer2DModel(
 sample_size=4, patch_size=2, in_channels=4, hidden_size=8, num_layers=1,
 num_refiner_layers=1, num_attention_heads=1, num_kv_heads=1,
 multiple_of=16, axes_dim_rope=(4, 2, 2), axes_lens=(32, 32, 32), cap_feat_dim=8,
).to(torch.bfloat16)
pipe = Lumina2Pipeline(transformer, FlowMatchEulerDiscreteScheduler(), None, None, None)
pipe.set_progress_bar_config(disable=True)

pipe(
 prompt=None,
 prompt_embeds=torch.randn(1, 4, 8, dtype=torch.float32),
 prompt_attention_mask=torch.ones(1, 4, dtype=torch.bool),
 guidance_scale=1.0,
 num_inference_steps=1,
 height=32,
 width=32,
 output_type="latent",
)

Relevant precedent:
Flux/Qwen-style prompt paths normalize prompt tensors before transformer use.

diffusers/src/diffusers/pipelines/qwenimage/pipeline_qwenimage.py

Lines 256 to 264 in 0f1abc4

	prompt_embeds = prompt_embeds[:, :max_sequence_length]
	_, seq_len, _ = prompt_embeds.shape
	prompt_embeds = prompt_embeds.repeat(1, num_images_per_prompt, 1)
	prompt_embeds = prompt_embeds.view(batch_size * num_images_per_prompt, seq_len, -1)
	if prompt_embeds_mask is not None:
	prompt_embeds_mask = prompt_embeds_mask[:, :max_sequence_length]
	prompt_embeds_mask = prompt_embeds_mask.repeat(1, num_images_per_prompt, 1)
	prompt_embeds_mask = prompt_embeds_mask.view(batch_size * num_images_per_prompt, seq_len)

Suggested fix:

dtype = self.transformer.dtype if self.transformer is not None else prompt_embeds.dtype
prompt_embeds = prompt_embeds.to(device=device, dtype=dtype)
if negative_prompt_embeds is not None:
 negative_prompt_embeds = negative_prompt_embeds.to(device=device, dtype=dtype)

Issue 4: Scheduler shift uses latent channels as image sequence length

Affected code:

diffusers/src/diffusers/pipelines/lumina2/pipeline_lumina2.py

Lines 697 to 716 in 0f1abc4

	# 5. Prepare timesteps
	sigmas = np.linspace(1.0, 1 / num_inference_steps, num_inference_steps) if sigmas is None else sigmas
	image_seq_len = latents.shape[1]
	mu = calculate_shift(
	image_seq_len,
	self.scheduler.config.get("base_image_seq_len", 256),
	self.scheduler.config.get("max_image_seq_len", 4096),
	self.scheduler.config.get("base_shift", 0.5),
	self.scheduler.config.get("max_shift", 1.15),
	)
	if XLA_AVAILABLE:
	timestep_device = "cpu"
	else:
	timestep_device = device
	timesteps, num_inference_steps = retrieve_timesteps(
	self.scheduler,
	num_inference_steps,
	timestep_device,
	sigmas=sigmas,
	mu=mu,

Problem:
image_seq_len = latents.shape[1] reads the channel dimension. Lumina2 latents are BCHW at this point, so the scheduler mu should be based on the number of image tokens after patching.

Impact:
Resolution-dependent timestep shifting is wrong. For a 1024x1024 image with default 16-channel latents and patch size 2, the code uses 16 instead of 4096.

Reproduction:

import torch
from diffusers import Lumina2Transformer2DModel

def calculate_shift(image_seq_len, base_seq_len=256, max_seq_len=4096, base_shift=0.5, max_shift=1.15):
 m = (max_shift - base_shift) / (max_seq_len - base_seq_len)
 return image_seq_len * m + (base_shift - m * base_seq_len)

transformer = Lumina2Transformer2DModel(
 sample_size=128, patch_size=2, in_channels=16, hidden_size=8, num_layers=0,
 num_refiner_layers=0, num_attention_heads=1, num_kv_heads=1,
 multiple_of=16, axes_dim_rope=(4, 2, 2), cap_feat_dim=8,
)
latents = torch.zeros(1, transformer.config.in_channels, 128, 128)
wrong = latents.shape[1]
correct = (latents.shape[2] // transformer.config.patch_size) * (latents.shape[3] // transformer.config.patch_size)
print(wrong, calculate_shift(wrong))
print(correct, calculate_shift(correct))

Relevant precedent:
Duplicate/related existing items: #12913 and #13272

Suggested fix:

patch_size = self.transformer.config.patch_size
image_seq_len = (latents.shape[2] // patch_size) * (latents.shape[3] // patch_size)

Issue 5: Lumina2 attention bypasses diffusers attention dispatch

Affected code:

diffusers/src/diffusers/models/transformers/transformer_lumina2.py

Lines 69 to 145 in 0f1abc4

	r"""
	Processor for implementing scaled dot-product attention (enabled by default if you're using PyTorch 2.0). This is
	used in the Lumina2Transformer2DModel model. It applies normalization and RoPE on query and key vectors.
	"""
	def __init__(self):
	if not hasattr(F, "scaled_dot_product_attention"):
	raise ImportError("AttnProcessor2_0 requires PyTorch 2.0, to use it, please upgrade PyTorch to 2.0.")
	def __call__(
	self,
	attn: Attention,
	hidden_states: torch.Tensor,
	encoder_hidden_states: torch.Tensor,
	attention_mask: torch.Tensor | None = None,
	image_rotary_emb: torch.Tensor | None = None,
	base_sequence_length: int | None = None,
	) -> torch.Tensor:
	batch_size, sequence_length, _ = hidden_states.shape
	# Get Query-Key-Value Pair
	query = attn.to_q(hidden_states)
	key = attn.to_k(encoder_hidden_states)
	value = attn.to_v(encoder_hidden_states)
	query_dim = query.shape[-1]
	inner_dim = key.shape[-1]
	head_dim = query_dim // attn.heads
	dtype = query.dtype
	# Get key-value heads
	kv_heads = inner_dim // head_dim
	query = query.view(batch_size, -1, attn.heads, head_dim)
	key = key.view(batch_size, -1, kv_heads, head_dim)
	value = value.view(batch_size, -1, kv_heads, head_dim)
	# Apply Query-Key Norm if needed
	if attn.norm_q is not None:
	query = attn.norm_q(query)
	if attn.norm_k is not None:
	key = attn.norm_k(key)
	# Apply RoPE if needed
	if image_rotary_emb is not None:
	query = apply_rotary_emb(query, image_rotary_emb, use_real=False)
	key = apply_rotary_emb(key, image_rotary_emb, use_real=False)
	query, key = query.to(dtype), key.to(dtype)
	# Apply proportional attention if true
	if base_sequence_length is not None:
	softmax_scale = math.sqrt(math.log(sequence_length, base_sequence_length)) * attn.scale
	else:
	softmax_scale = attn.scale
	# perform Grouped-qurey Attention (GQA)
	n_rep = attn.heads // kv_heads
	if n_rep >= 1:
	key = key.unsqueeze(3).repeat(1, 1, 1, n_rep, 1).flatten(2, 3)
	value = value.unsqueeze(3).repeat(1, 1, 1, n_rep, 1).flatten(2, 3)
	# scaled_dot_product_attention expects attention_mask shape to be
	# (batch, heads, source_length, target_length)
	if attention_mask is not None:
	attention_mask = attention_mask.bool().view(batch_size, 1, 1, -1)
	query = query.transpose(1, 2)
	key = key.transpose(1, 2)
	value = value.transpose(1, 2)
	hidden_states = F.scaled_dot_product_attention(
	query, key, value, attn_mask=attention_mask, scale=softmax_scale
	)
	hidden_states = hidden_states.transpose(1, 2).reshape(batch_size, -1, attn.heads * head_dim)
	hidden_states = hidden_states.type_as(query)

Problem:
Lumina2AttnProcessor2_0 calls F.scaled_dot_product_attention directly and has no _attention_backend / _parallel_config. ModelMixin.set_attention_backend() cannot configure these processors.

Impact:
Lumina2 misses the current attention backend system, including backend selection and context-parallel-compatible dispatch.

Reproduction:

from diffusers import Lumina2Transformer2DModel

model = Lumina2Transformer2DModel(
 sample_size=16, patch_size=2, in_channels=4, hidden_size=24,
 num_layers=1, num_refiner_layers=1, num_attention_heads=3,
 num_kv_heads=1, multiple_of=2, axes_dim_rope=(4, 2, 2),
 axes_lens=(128, 128, 128), cap_feat_dim=32,
)
print([(m.processor.__class__.__name__, hasattr(m.processor, "_attention_backend")) for m in model.modules() if hasattr(m, "processor")])
model.set_attention_backend("native")
print([getattr(m.processor, "_attention_backend", "MISSING") for m in model.modules() if hasattr(m, "processor")])

Relevant precedent:
Flux processors declare _attention_backend and call dispatch_attention_fn.

diffusers/src/diffusers/models/transformers/transformer_flux.py

Lines 75 to 125 in 0f1abc4

	class FluxAttnProcessor:
	_attention_backend = None
	_parallel_config = None
	def __init__(self):
	if not hasattr(F, "scaled_dot_product_attention"):
	raise ImportError(f"{self.__class__.__name__} requires PyTorch 2.0. Please upgrade your pytorch version.")
	def __call__(
	self,
	attn: "FluxAttention",
	hidden_states: torch.Tensor,
	encoder_hidden_states: torch.Tensor = None,
	attention_mask: torch.Tensor | None = None,
	image_rotary_emb: torch.Tensor | None = None,
	) -> torch.Tensor:
	query, key, value, encoder_query, encoder_key, encoder_value = _get_qkv_projections(
	attn, hidden_states, encoder_hidden_states
	)
	query = query.unflatten(-1, (attn.heads, -1))
	key = key.unflatten(-1, (attn.heads, -1))
	value = value.unflatten(-1, (attn.heads, -1))
	query = attn.norm_q(query)
	key = attn.norm_k(key)
	if attn.added_kv_proj_dim is not None:
	encoder_query = encoder_query.unflatten(-1, (attn.heads, -1))
	encoder_key = encoder_key.unflatten(-1, (attn.heads, -1))
	encoder_value = encoder_value.unflatten(-1, (attn.heads, -1))
	encoder_query = attn.norm_added_q(encoder_query)
	encoder_key = attn.norm_added_k(encoder_key)
	query = torch.cat([encoder_query, query], dim=1)
	key = torch.cat([encoder_key, key], dim=1)
	value = torch.cat([encoder_value, value], dim=1)
	if image_rotary_emb is not None:
	query = apply_rotary_emb(query, image_rotary_emb, sequence_dim=1)
	key = apply_rotary_emb(key, image_rotary_emb, sequence_dim=1)
	hidden_states = dispatch_attention_fn(
	query,
	key,
	value,
	attn_mask=attention_mask,
	backend=self._attention_backend,
	parallel_config=self._parallel_config,
	)

Suggested fix:
Port Lumina2 attention to the current pattern: define a Lumina2 attention module with AttentionModuleMixin, give the processor _attention_backend and _parallel_config, and call dispatch_attention_fn on (batch, sequence, heads, head_dim) query/key/value tensors.

Issue 6: RoPE precomputes complex128 tensors by default

Affected code:

diffusers/src/diffusers/models/transformers/transformer_lumina2.py

Lines 243 to 249 in 0f1abc4

	def _precompute_freqs_cis(self, axes_dim: list[int], axes_lens: list[int], theta: int) -> list[torch.Tensor]:
	freqs_cis = []
	freqs_dtype = torch.float32 if torch.backends.mps.is_available() else torch.float64
	for i, (d, e) in enumerate(zip(axes_dim, axes_lens)):
	emb = get_1d_rotary_pos_embed(d, e, theta=self.theta, freqs_dtype=freqs_dtype)
	freqs_cis.append(emb)
	return freqs_cis

Problem:
RoPE precompute uses torch.float64 unless MPS is globally available, which produces torch.complex128 frequency tensors. The review rules disallow unconditional float64 and call out NPU/MPS compatibility.

Impact:
This adds unnecessary memory/cast overhead and can break unsupported float64 backends.

Reproduction:

from diffusers import Lumina2Transformer2DModel

model = Lumina2Transformer2DModel(
 sample_size=16, patch_size=2, in_channels=4, hidden_size=24,
 num_layers=1, num_refiner_layers=1, num_attention_heads=3,
 num_kv_heads=1, multiple_of=2, axes_dim_rope=(4, 2, 2),
 axes_lens=(128, 128, 128), cap_feat_dim=32,
)
print([freqs.dtype for freqs in model.rope_embedder.freqs_cis])
# [torch.complex128, torch.complex128, torch.complex128]

Relevant precedent:
Flux gates MPS and NPU explicitly for RoPE dtype.

diffusers/src/diffusers/models/transformers/transformer_flux.py

Lines 505 to 518 in 0f1abc4

	pos = ids.float()
	is_mps = ids.device.type == "mps"
	is_npu = ids.device.type == "npu"
	freqs_dtype = torch.float32 if (is_mps or is_npu) else torch.float64
	for i in range(n_axes):
	cos, sin = get_1d_rotary_pos_embed(
	self.axes_dim[i],
	pos[:, i],
	theta=self.theta,
	repeat_interleave_real=True,
	use_real=True,
	freqs_dtype=freqs_dtype,
	)
	cos_out.append(cos)

Suggested fix:

freqs_dtype = torch.float32

Issue 7: RoPE position construction breaks torch.compile(fullgraph=True)

Affected code:

diffusers/src/diffusers/models/transformers/transformer_lumina2.py

Lines 263 to 291 in 0f1abc4

	def forward(self, hidden_states: torch.Tensor, attention_mask: torch.Tensor):
	batch_size, channels, height, width = hidden_states.shape
	p = self.patch_size
	post_patch_height, post_patch_width = height // p, width // p
	image_seq_len = post_patch_height * post_patch_width
	device = hidden_states.device
	encoder_seq_len = attention_mask.shape[1]
	l_effective_cap_len = attention_mask.sum(dim=1).tolist()
	seq_lengths = [cap_seq_len + image_seq_len for cap_seq_len in l_effective_cap_len]
	max_seq_len = max(seq_lengths)
	# Create position IDs
	position_ids = torch.zeros(batch_size, max_seq_len, 3, dtype=torch.int32, device=device)
	for i, (cap_seq_len, seq_len) in enumerate(zip(l_effective_cap_len, seq_lengths)):
	# add caption position ids
	position_ids[i, :cap_seq_len, 0] = torch.arange(cap_seq_len, dtype=torch.int32, device=device)
	position_ids[i, cap_seq_len:seq_len, 0] = cap_seq_len
	# add image position ids
	row_ids = (
	torch.arange(post_patch_height, dtype=torch.int32, device=device)
	.view(-1, 1)
	.repeat(1, post_patch_width)
	.flatten()
	)
	col_ids = (
	torch.arange(post_patch_width, dtype=torch.int32, device=device)

Problem:
attention_mask.sum(dim=1).tolist() moves tensor values into Python, then Python max() and per-sample loops use those values for shapes/slices. This violates the model review rule to avoid graph breaks in forward implementations.

Impact:
torch.compile(..., fullgraph=True) cannot compile the model, and device execution pays synchronization overhead.

Reproduction:

import torch
from diffusers import Lumina2Transformer2DModel

model = Lumina2Transformer2DModel(
 sample_size=16, patch_size=2, in_channels=4, hidden_size=24,
 num_layers=1, num_refiner_layers=1, num_attention_heads=3,
 num_kv_heads=1, multiple_of=2, axes_dim_rope=(4, 2, 2),
 axes_lens=(128, 128, 128), cap_feat_dim=32,
).eval()

compiled = torch.compile(model, fullgraph=True, backend="eager")
compiled(
 hidden_states=torch.randn(1, 4, 16, 16),
 timestep=torch.rand(1),
 encoder_hidden_states=torch.randn(1, 16, 32),
 encoder_attention_mask=torch.ones(1, 16, dtype=torch.bool),
)

Relevant precedent:
The model review rules explicitly require avoiding graph breaks in forward implementations.

Suggested fix:
Vectorize position-id and joint-sequence assembly so mask lengths stay as tensors. If variable effective caption lengths are required, use tensor masks/scatter operations rather than .tolist() and Python-derived allocation sizes.

Issue 8: Slow end-to-end Lumina2 pipeline coverage is missing

Affected code:

diffusers/tests/pipelines/lumina2/test_pipeline_lumina2.py

Lines 16 to 117 in 0f1abc4

	class Lumina2PipelineFastTests(unittest.TestCase, PipelineTesterMixin):
	pipeline_class = Lumina2Pipeline
	params = frozenset(
	[
	"prompt",
	"height",
	"width",
	"guidance_scale",
	"negative_prompt",
	"prompt_embeds",
	"negative_prompt_embeds",
	]
	)
	batch_params = frozenset(["prompt", "negative_prompt"])
	required_optional_params = frozenset(
	[
	"num_inference_steps",
	"generator",
	"latents",
	"return_dict",
	"callback_on_step_end",
	"callback_on_step_end_tensor_inputs",
	]
	)
	supports_dduf = False
	test_xformers_attention = False
	test_layerwise_casting = True
	def get_dummy_components(self):
	torch.manual_seed(0)
	transformer = Lumina2Transformer2DModel(
	sample_size=4,
	patch_size=2,
	in_channels=4,
	hidden_size=8,
	num_layers=2,
	num_attention_heads=1,
	num_kv_heads=1,
	multiple_of=16,
	ffn_dim_multiplier=None,
	norm_eps=1e-5,
	scaling_factor=1.0,
	axes_dim_rope=[4, 2, 2],
	cap_feat_dim=8,
	)
	torch.manual_seed(0)
	vae = AutoencoderKL(
	sample_size=32,
	in_channels=3,
	out_channels=3,
	block_out_channels=(4,),
	layers_per_block=1,
	latent_channels=4,
	norm_num_groups=1,
	use_quant_conv=False,
	use_post_quant_conv=False,
	shift_factor=0.0609,
	scaling_factor=1.5035,
	)
	scheduler = FlowMatchEulerDiscreteScheduler()
	tokenizer = AutoTokenizer.from_pretrained("hf-internal-testing/dummy-gemma")
	torch.manual_seed(0)
	config = Gemma2Config(
	head_dim=4,
	hidden_size=8,
	intermediate_size=8,
	num_attention_heads=2,
	num_hidden_layers=2,
	num_key_value_heads=2,
	sliding_window=2,
	)
	text_encoder = Gemma2Model(config)
	components = {
	"transformer": transformer,
	"vae": vae.eval(),
	"scheduler": scheduler,
	"text_encoder": text_encoder,
	"tokenizer": tokenizer,
	}
	return components
	def get_dummy_inputs(self, device, seed=0):
	if str(device).startswith("mps"):
	generator = torch.manual_seed(seed)
	else:
	generator = torch.Generator(device="cpu").manual_seed(seed)
	inputs = {
	"prompt": "A painting of a squirrel eating a burger",
	"generator": generator,
	"num_inference_steps": 2,
	"guidance_scale": 5.0,
	"height": 32,
	"width": 32,
	"output_type": "np",
	}
	return inputs

diffusers/tests/single_file/test_lumina2_transformer.py

Lines 30 to 38 in 0f1abc4

	class TestLumina2Transformer2DModelSingleFile(SingleFileModelTesterMixin):
	model_class = Lumina2Transformer2DModel
	ckpt_path = "https://huggingface.co/Comfy-Org/Lumina_Image_2.0_Repackaged/blob/main/split_files/diffusion_models/lumina_2_model_bf16.safetensors"
	alternate_keys_ckpt_paths = [
	"https://huggingface.co/Comfy-Org/Lumina_Image_2.0_Repackaged/blob/main/split_files/diffusion_models/lumina_2_model_bf16.safetensors"
	]
	repo_id = "Alpha-VLLM/Lumina-Image-2.0"
	subfolder = "transformer"

Problem:
Lumina2 has fast dummy pipeline/model tests, LoRA tests, DreamBooth example tests, and a single-file transformer loading test, but no slow end-to-end pipeline test with the real Alpha-VLLM/Lumina-Image-2.0 components and an expected output slice.

Impact:
Scheduler/parity bugs such as the mu issue, dtype behavior, and real-checkpoint prompt/decoder behavior can ship without a slow regression signal.

Reproduction:

from pathlib import Path

matches = []
for path in Path("tests").rglob("*lumina2*.py"):
 text = path.read_text(encoding="utf-8")
 if "@slow" in text or "@nightly" in text:
 matches.append(str(path))
print(matches)
# []

Relevant precedent:
Flux has a real-checkpoint slow pipeline test class.

diffusers/tests/pipelines/flux/test_pipeline_flux.py

Lines 238 to 275 in 0f1abc4

	@nightly
	@require_big_accelerator
	class FluxPipelineSlowTests(unittest.TestCase):
	pipeline_class = FluxPipeline
	repo_id = "black-forest-labs/FLUX.1-schnell"
	def setUp(self):
	super().setUp()
	gc.collect()
	backend_empty_cache(torch_device)
	def tearDown(self):
	super().tearDown()
	gc.collect()
	backend_empty_cache(torch_device)
	def get_inputs(self, device, seed=0):
	generator = torch.Generator(device="cpu").manual_seed(seed)
	prompt_embeds = torch.load(
	hf_hub_download(repo_id="diffusers/test-slices", repo_type="dataset", filename="flux/prompt_embeds.pt")
	).to(torch_device)
	pooled_prompt_embeds = torch.load(
	hf_hub_download(
	repo_id="diffusers/test-slices", repo_type="dataset", filename="flux/pooled_prompt_embeds.pt"
	)
	).to(torch_device)
	return {
	"prompt_embeds": prompt_embeds,
	"pooled_prompt_embeds": pooled_prompt_embeds,
	"num_inference_steps": 2,
	"guidance_scale": 0.0,
	"max_sequence_length": 256,
	"output_type": "np",
	"generator": generator,
	}
	def test_flux_inference(self):

Suggested fix:
Add a Lumina2PipelineSlowTests class under tests/pipelines/lumina2/test_pipeline_lumina2.py that loads the public checkpoint or cached test slices, runs a tiny deterministic inference, and asserts an expected image slice.