QwenImagepipeline imcompatable with group offloading with record_stream=True

[chaowenguo]

Describe the bug

I use group offloading with QwenImagepipeline, it shows a lot of blocks are not used in forward. And the final image is a mess. I try enable_sequential_cpu_offload is working fine. I believe it is the error from group offloading.

Reproduction

from diffusers import DiffusionPipeline, FlowMatchEulerDiscreteScheduler
import torch 
import math

# From https://github.com/ModelTC/Qwen-Image-Lightning/blob/342260e8f5468d2f24d084ce04f55e101007118b/generate_with_diffusers.py#L82C9-L97C10
scheduler_config = {
 "base_image_seq_len": 256,
 "base_shift": math.log(3), # We use shift=3 in distillation
 "invert_sigmas": False,
 "max_image_seq_len": 8192,
 "max_shift": math.log(3), # We use shift=3 in distillation
 "num_train_timesteps": 1000,
 "shift": 1.0,
 "shift_terminal": None, # set shift_terminal to None
 "stochastic_sampling": False,
 "time_shift_type": "exponential",
 "use_beta_sigmas": False,
 "use_dynamic_shifting": True,
 "use_exponential_sigmas": False,
 "use_karras_sigmas": False,
}
scheduler = FlowMatchEulerDiscreteScheduler.from_config(scheduler_config)
pipe = DiffusionPipeline.from_pretrained(
 "Qwen/Qwen-Image", scheduler=scheduler, torch_dtype=torch.bfloat16
)
pipe.load_lora_weights(
 "lightx2v/Qwen-Image-Lightning", weight_name="Qwen-Image-Lightning-8steps-V1.0.safetensors"
)
onload_device = torch.device("cuda")
pipe.transformer.enable_group_offload(onload_device=onload_device, offload_type="leaf_level", use_stream=True, record_stream=True, non_blocking=True, low_cpu_mem_usage=True)
diffusers.hooks.apply_group_offloading(pipe.text_encoder, onload_device=onload_device, offload_type="leaf_level", use_stream=True, record_stream=True, non_blocking=True, low_cpu_mem_usage=True)

prompt = "a tiny astronaut hatching from an egg on the moon, Ultra HD, 4K, cinematic composition."
negative_prompt = " "
image = pipe(
 prompt=prompt,
 negative_prompt=negative_prompt,
 width=1024,
 height=1024,
 num_inference_steps=8,
 true_cfg_scale=1.0,
 generator=torch.manual_seed(0),
).images[0]
image.save("qwen_fewsteps.png")

Logs

System Info

ubuntu22.04-cuda12.4.0-py311-torch2.8.0-diffusers0.35.2

Who can help?

@yiyixuxu @DN6

[DN6]

@chaowenguo0 do you observe the same issue if you disable LoRA loading?

[chaowenguo]

yes

[asomoza]

after digging a little more into this issue, the warning is not with the lora or the transformer model, is with the text encoder, this code works ok:

import torch

from diffusers import QwenImagePipeline


device = torch.device("cuda")

pipe = QwenImagePipeline.from_pretrained("Qwen/Qwen-Image", torch_dtype=torch.bfloat16)
pipe.transformer.enable_group_offload(
 onload_device=device, offload_type="leaf_level", use_stream=True, non_blocking=True, low_cpu_mem_usage=True
)
pipe.to(device)

prompt = "a tiny astronaut hatching from an egg on the moon, Ultra HD, 4K, cinematic composition."
negative_prompt = " "
image = pipe(
 prompt=prompt,
 negative_prompt=negative_prompt,
 width=1024,
 height=1024,
 num_inference_steps=50,
 true_cfg_scale=4.0,
 generator=torch.manual_seed(42),
).images[0]
image.save("qwen_image_output.png")

and generates this image:

but this code (adding group offloading to the text encoder) gives the warning of blocks not used in the forward:

It seems like some layers were not executed during the forward pass. This may lead to problems when applying lazy prefetching with automatic tracing and lead to device-mismatch related errors. Please make sure that all layers are executed during the forward pass. The following layers were not executed:
unexecuted_layers=['model.visual.blocks.23.norm2', 'model.visual.merger.mlp.2',...]

import torch

from diffusers import QwenImagePipeline
from diffusers.hooks import apply_group_offloading


device = torch.device("cuda")

pipe = QwenImagePipeline.from_pretrained("Qwen/Qwen-Image", torch_dtype=torch.bfloat16)
pipe.transformer.enable_group_offload(
 onload_device=device, offload_type="leaf_level", use_stream=True, non_blocking=True, low_cpu_mem_usage=True
)
apply_group_offloading(
 pipe.text_encoder,
 onload_device=device,
 offload_type="leaf_level",
 use_stream=True,
 non_blocking=True,
 low_cpu_mem_usage=True,
)
pipe.to(device)

prompt = "a tiny astronaut hatching from an egg on the moon, Ultra HD, 4K, cinematic composition."
negative_prompt = " "
image = pipe(
 prompt=prompt,
 negative_prompt=negative_prompt,
 width=1024,
 height=1024,
 num_inference_steps=50,
 true_cfg_scale=4.0,
 generator=torch.manual_seed(42),
).images[0]
image.save("qwen_image_output.png")

but in fact gives the exact same image without any difference in my eyes:

@chaowenguo0 , the blocks warning and group offloading doesn't seem to have anything to do with the "final image is a mess" so the issue is not correct, this is just a guess (since you didn't post the image) but probably what's happening is that the lora isn't loading and the model is just generating an image with no cfg and low steps like this one:

[chaowenguo]

so I just ignore the warning?

…

On Wed, Nov 19, 2025 at 3:47 PM Álvaro Somoza ***@***.***> wrote: *asomoza* left a comment (huggingface/diffusers#12673) <#12673 (comment)> after digging a little more into this issue, the issue is not with the lora or the transformer model, is with the text encoder, this code works ok: import torch from diffusers import QwenImagePipeline device = torch.device("cuda") pipe = QwenImagePipeline.from_pretrained("Qwen/Qwen-Image", torch_dtype=torch.bfloat16)pipe.transformer.enable_group_offload( onload_device=device, offload_type="leaf_level", use_stream=True, non_blocking=True, low_cpu_mem_usage=True )pipe.to(device) prompt = "a tiny astronaut hatching from an egg on the moon, Ultra HD, 4K, cinematic composition."negative_prompt = " "image = pipe( prompt=prompt, negative_prompt=negative_prompt, width=1024, height=1024, num_inference_steps=50, true_cfg_scale=4.0, generator=torch.manual_seed(42), ).images[0]image.save("qwen_image_output.png") and generates this image: qwen_image_output.png (view on web) <https://github.com/user-attachments/assets/a3db4e86-6bf7-4b1b-ae3f-9266d1e19978> but this code (adding group offloading to the text encoder) gives the warning of blocks not used in the forward: It seems like some layers were not executed during the forward pass. This may lead to problems when applying lazy prefetching with automatic tracing and lead to device-mismatch related errors. Please make sure that all layers are executed during the forward pass. The following layers were not executed: unexecuted_layers=['model.visual.blocks.23.norm2', 'model.visual.merger.mlp.2',...] import torch from diffusers import QwenImagePipelinefrom diffusers.hooks import apply_group_offloading device = torch.device("cuda") pipe = QwenImagePipeline.from_pretrained("Qwen/Qwen-Image", torch_dtype=torch.bfloat16)pipe.transformer.enable_group_offload( onload_device=device, offload_type="leaf_level", use_stream=True, non_blocking=True, low_cpu_mem_usage=True )apply_group_offloading( pipe.text_encoder, onload_device=device, offload_type="leaf_level", use_stream=True, non_blocking=True, low_cpu_mem_usage=True, )pipe.to(device) prompt = "a tiny astronaut hatching from an egg on the moon, Ultra HD, 4K, cinematic composition."negative_prompt = " "image = pipe( prompt=prompt, negative_prompt=negative_prompt, width=1024, height=1024, num_inference_steps=50, true_cfg_scale=4.0, generator=torch.manual_seed(42), ).images[0]image.save("qwen_image_output.png") but in fact gives the exact same image without any difference in my eyes: qwen_image_output_2.png (view on web) <https://github.com/user-attachments/assets/9cd16549-04a0-4e9e-a4e9-7944006c395f> @chaowenguo0 <https://github.com/chaowenguo0> , the blocks warning and group offloading doesn't seem to have anything to do with the "final image is a mess" so the issue is not correct, this is just a guess (since you didn't post the image) but probably what's happening is that the lora isn't loading and the model is just generating an image with no cfg and low steps like this one: qwen_image_output_3.png (view on web) <https://github.com/user-attachments/assets/6faa9299-18d4-4125-98a5-8a68e695ede1> — Reply to this email directly, view it on GitHub <#12673 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BWUG5QDX5LPO7V25SD3GJDL35T6SVAVCNFSM6AAAAACMJRFR7SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTKNJVGEYDGOBQGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[asomoza]

I'm not saying that, what I'm saying is that the issue is not correct, you need to first correctly identify the issue you're having and then edit the issue accordingly with a minimal reproducible code snippet.

what's your issue? the warning or the bad image? The title of the issue is not correct since it is QwenImagepipeline incompatible with group offloading and group offload works with this pipeline but it throws a warning and also your code has the lora loading which is not relevant and just adds noise, unless your problem is with the lora.

[chaowenguo]

@asomoza, it seem the bug is from record_stream=True. without record_stream=True I get

with record_stream=True I get

[KimbingNg]

I believe this issue DO exists! I encountered the same issue with HunyuanVideo-1.5 when record_stream is enabled.
After further investigation, I identified the root cause: record_stream isn’t functioning as expected. This causes the offload_to_memory function to update param.data without waiting for the current stream to complete (the forward pass). I’ve submitted a PR to fix this in #12721.

The root cause
The current_stream() call inside _transfer_tensor_to_device is under with torch.cuda.stream(self.stream). So it is not returning the correct default stream to call record_stream on !