export_to_video causes: ValueError: too many values to unpack (expected 3)

[tin2tin]

Describe the bug

export_to_video causes: ValueError: too many values to unpack (expected 3)

I know that some work has been done here, but the error persists in 0.26.2, even though this patch has been committed some time ago : #6715

Reproduction

Run the code from: https://huggingface.co/docs/diffusers/v0.26.2/en/api/pipelines/text_to_video#cerspensezeroscopev2576w--cerspensezeroscopev2xl

import torch
from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler
from diffusers.utils import export_to_video
from PIL import Image

pipe = DiffusionPipeline.from_pretrained("cerspense/zeroscope_v2_576w", torch_dtype=torch.float16)
pipe.enable_model_cpu_offload()

# memory optimization
pipe.unet.enable_forward_chunking(chunk_size=1, dim=1)
pipe.enable_vae_slicing()

prompt = "Darth Vader surfing a wave"
video_frames = pipe(prompt, num_frames=24).frames
video_path = export_to_video(video_frames)
video_path

Logs

vae\diffusion_pytorch_model.safetensors not found
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 5/5 [00:06<00:00, 1.20s/it]
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [03:53<00:00, 4.67s/it]
Traceback (most recent call last):
 File "...\AppData\Roaming\Python\Python311\site-packages\diffusers\utils\export_utils.py", line 135, in export_to_video
 h, w, c = video_frames[0].shape
 ^^^^^^^
ValueError: too many values to unpack (expected 3)
Error: Python: Traceback (most recent call last):
 File "...\AppData\Roaming\Python\Python311\site-packages\diffusers\utils\export_utils.py", line 135, in export_to_video
 h, w, c = video_frames[0].shape
 ^^^^^^^
ValueError: too many values to unpack (expected 3)

System Info

Win 11
Diffusers: 0.26.2

Who can help?

@DN6 @sayakpaul

[tolgacangoz]

Hi @tin2tin! Could you please give video_frames[0] to export_to_video function:

video_path = export_to_video(video_frames[0])

[tin2tin]

Yes, no errors with [0], but you only get a freeze-frame of course.

[DN6]

@tin2tin The following snippet is working for me. Note that using frames[0] will return the first video in the batch. You shouldn't see just a single frame.

import torch
from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler
from diffusers.utils import export_to_video
from PIL import Image

pipe = DiffusionPipeline.from_pretrained("cerspense/zeroscope_v2_576w", torch_dtype=torch.float16)
pipe.enable_model_cpu_offload()

# memory optimization
pipe.unet.enable_forward_chunking(chunk_size=1, dim=1)
pipe.enable_vae_slicing()

prompt = "Darth Vader surfing a wave"
video_frames = pipe(prompt, num_frames=24).frames[0]
video_path = export_to_video(video_frames, fps=10, output_video_path="vid-export.mp4")

[tin2tin]

@DN6 Yes. Thank you. This is working. Maybe the documentation needs a check-up, to ensure that this is inserted where needed.

[yiyixuxu]

@tin2tin
I think we updated the doc for this recently - happy to accept a PR if you spotted places that we missed:)