skyreels_v2 model/pipeline review
Commit tested: 0f1abc4ae8b0eb2a3b40e82a310507281144c423
Review performed against the repository review rules. Reviewed public imports/lazy loading, transformer config/attention/runtime paths, all SkyReels-V2 pipelines, docs, examples, and tests. Duplicate searches found an existing duplicate only for the optional ftfy issue: #13112 and PR #13113.
Top-level imports passed in .venv. Full pytest collection could not run because this .venv torch build lacks torch._C._distributed_c10d, which the shared test mixins import through FSDP.
Issue 1: Diffusion-forcing schedules drop tail latent frames
Affected code:
| # Each block contains causal_block_size frames that are processed together |
| # E.g.: 25 frames ÷ 5 = 5 blocks total |
| num_blocks=num_latent_frames//causal_block_size |
| base_num_blocks=base_num_latent_frames//causal_block_size |
| ifcausal_block_size>1: |
| # Expand each block to causal_block_size frames |
| step_update_mask=step_update_mask.unsqueeze(-1).repeat(1, 1, causal_block_size).flatten(1).contiguous() |
| step_index=step_index.unsqueeze(-1).repeat(1, 1, causal_block_size).flatten(1).contiguous() |
| step_matrix=step_matrix.unsqueeze(-1).repeat(1, 1, causal_block_size).flatten(1).contiguous() |
| # Scale intervals from block-level to frame-level |
| valid_interval= [(s*causal_block_size, e*causal_block_size) fors, einvalid_interval] |
| ifself.config.num_frame_per_block>1: |
| block_num=post_patch_num_frames//self.config.num_frame_per_block |
| range_tensor=torch.arange(block_num, device=hidden_states.device).repeat_interleave( |
| self.config.num_frame_per_block |
| ) |
| causal_mask=range_tensor.unsqueeze(0) <=range_tensor.unsqueeze(1) # f, f |
| causal_mask=causal_mask.view(post_patch_num_frames, 1, 1, post_patch_num_frames, 1, 1) |
| causal_mask=causal_mask.repeat( |
| 1, post_patch_height, post_patch_width, 1, post_patch_height, post_patch_width |
| ) |
| causal_mask=causal_mask.reshape( |
| post_patch_num_frames*post_patch_height*post_patch_width, |
| post_patch_num_frames*post_patch_height*post_patch_width, |
| ) |
| causal_mask=causal_mask.unsqueeze(0).unsqueeze(0) |
Problem:
generate_timestep_matrix() floors num_latent_frames // causal_block_size and later expands back to full frames. If the latent frame count is not divisible by the block size, the remainder is omitted. The documented 720P setting base_num_frames=121 gives 31 latent frames with temporal scale 4, and causal_block_size=5 covers only 30.
Impact:
The final latent frame can remain at initial noise or be skipped by the update schedule. Direct transformer calls with num_frame_per_block > 1 also build masks assuming exact divisibility.
Reproduction:
importtorchfromdiffusersimportSkyReelsV2DiffusionForcingPipelinenum_latent_frames= (121-1) //4+1step_matrix, _, update_mask, intervals=SkyReelsV2DiffusionForcingPipeline.generate_timestep_matrix(
None, num_latent_frames, torch.arange(4), num_latent_frames, ar_step=0, causal_block_size=5
)
print(num_latent_frames, step_matrix.shape[1], update_mask.shape[1], intervals[-1])
# 31 30 30 (0, 30)
Relevant precedent:
The docs recommend the 121-frame 720P setting with causal block sizing here:
| num_frames=97, |
| base_num_frames=97, # 121 for 720P |
| ar_step=5, # Controls asynchronous inference (0 for synchronous mode) |
| causal_block_size=5, # Number of frames in each block for asynchronous processing |
| overlap_history=None, # Number of frames to overlap for smooth transitions in long videos; 17 for long video generations |
Suggested fix:
Use ceil block counts and crop expanded tensors back to num_latent_frames. In the model mask, derive block IDs per actual frame instead of repeat-interleaving a floored block count:
frame_ids=torch.arange(post_patch_num_frames, device=hidden_states.device)
block_ids=torch.div(frame_ids, self.config.num_frame_per_block, rounding_mode="floor")
causal_mask=block_ids.unsqueeze(0) <=block_ids.unsqueeze(1)
Issue 2: Overlap noise uses global indices on local windows
Affected code:
| ifaddnoise_condition>0andvalid_interval_start<prefix_video_latents_frames: |
| noise_factor=0.001*addnoise_condition |
| latent_model_input[:, :, valid_interval_start:prefix_video_latents_frames, :, :] = ( |
| latent_model_input[:, :, valid_interval_start:prefix_video_latents_frames, :, :] |
| * (1.0-noise_factor) |
| +torch.randn_like( |
| latent_model_input[:, :, valid_interval_start:prefix_video_latents_frames, :, :] |
| ) |
| *noise_factor |
| ) |
| timestep[:, valid_interval_start:prefix_video_latents_frames] =addnoise_condition |
| ifaddnoise_condition>0andvalid_interval_start<prefix_video_latents_frames: |
| noise_factor=0.001*addnoise_condition |
| latent_model_input[:, :, valid_interval_start:prefix_video_latents_frames, :, :] = ( |
| latent_model_input[:, :, valid_interval_start:prefix_video_latents_frames, :, :] |
| * (1.0-noise_factor) |
| +torch.randn_like( |
| latent_model_input[:, :, valid_interval_start:prefix_video_latents_frames, :, :] |
| ) |
| *noise_factor |
| ) |
| timestep[:, valid_interval_start:prefix_video_latents_frames] =addnoise_condition |
| ifaddnoise_condition>0andvalid_interval_start<prefix_video_latents_frames: |
| noise_factor=0.001*addnoise_condition |
| latent_model_input[:, :, valid_interval_start:prefix_video_latents_frames, :, :] = ( |
| latent_model_input[:, :, valid_interval_start:prefix_video_latents_frames, :, :] |
| * (1.0-noise_factor) |
| +torch.randn_like( |
| latent_model_input[:, :, valid_interval_start:prefix_video_latents_frames, :, :] |
| ) |
| *noise_factor |
| ) |
| timestep[:, valid_interval_start:prefix_video_latents_frames] =addnoise_condition |
Problem:
latent_model_input is already sliced to valid_interval_start:valid_interval_end, but the overlap-noise branch slices it again with global indices. It also uses torch.randn_like() directly, bypassing the user-provided generator.
Impact:
Long-video overlap conditioning is noised on the wrong local frames, and addnoise_condition > 0 is not reproducible from the pipeline generator.
Reproduction:
importtorchprefix_video_latents_frames=10valid_interval_start=5valid_interval_end=15latent_model_input=torch.arange(valid_interval_start, valid_interval_end).view(1, 1, -1, 1, 1).float()
current=latent_model_input.clone()
current[:, :, valid_interval_start:prefix_video_latents_frames] =-1expected=latent_model_input.clone()
expected[:, :, : prefix_video_latents_frames-valid_interval_start] =-1print(current.flatten().tolist())
print(expected.flatten().tolist())
Relevant precedent:
Other diffusers latent preparation uses randn_tensor(..., generator=generator) for all user-visible randomness.
Suggested fix:
local_prefix_end=min(prefix_video_latents_frames, valid_interval_end) -valid_interval_startifaddnoise_condition>0andlocal_prefix_end>0:
noise_factor=0.001*addnoise_conditionprefix=latent_model_input[:, :, :local_prefix_end]
noise=randn_tensor(prefix.shape, generator=generator, device=prefix.device, dtype=prefix.dtype)
latent_model_input[:, :, :local_prefix_end] =prefix* (1.0-noise_factor) +noise*noise_factortimestep[:, :local_prefix_end] =addnoise_condition
Issue 3: causal_block_size mutates serialized transformer config
Affected code:
| ifcausal_block_sizeisNone: |
| causal_block_size=self.transformer.config.num_frame_per_block |
| else: |
| self.transformer._set_ar_attention(causal_block_size) |
| def_set_ar_attention(self, causal_block_size: int): |
| self.register_to_config(num_frame_per_block=causal_block_size) |
Problem:
A per-call pipeline option calls self.transformer._set_ar_attention(causal_block_size), and _set_ar_attention() writes into the model config via register_to_config.
Impact:
One async call changes later calls that omit causal_block_size, and save_pretrained() will persist that call-time setting.
Reproduction:
fromdiffusersimportSkyReelsV2Transformer3DModelm=SkyReelsV2Transformer3DModel(
patch_size=(1, 2, 2), num_attention_heads=2, attention_head_dim=12,
in_channels=4, out_channels=4, text_dim=16, freq_dim=256,
ffn_dim=32, num_layers=1, rope_max_seq_len=32,
)
print(m.config.num_frame_per_block)
m._set_ar_attention(5)
print(m.config.num_frame_per_block)
Relevant precedent:
Pipeline call arguments should not silently rewrite loadable model config.
Suggested fix:
Pass the block size as a runtime transformer forward argument, or temporarily restore the original config in a try/finally around the denoising loop.
Issue 4: I2V input validation rejects valid image batches and accepts unusable embeds
Affected code:
| ifimageisnotNoneandnotisinstance(image, torch.Tensor) andnotisinstance(image, PIL.Image.Image): |
| raiseValueError(f"`image` has to be of type `torch.Tensor` or `PIL.Image.Image` but is {type(image)}") |
| ifimageisnotNoneandimage_embedsisnotNone: |
| raiseValueError( |
| f"Cannot forward both `image`: {image} and `image_embeds`: {image_embeds}. Please make sure to" |
| " only forward one of the two." |
| ) |
| ifimageisNoneandimage_embedsisNone: |
| raiseValueError( |
| "Provide either `image` or `image_embeds`. Cannot leave both `image` and `image_embeds` undefined." |
| ) |
| ifimageisnotNoneandnotisinstance(image, torch.Tensor) andnotisinstance(image, PIL.Image.Image): |
| raiseValueError(f"`image` has to be of type `torch.Tensor` or `PIL.Image.Image` but is {type(image)}") |
| image=self.video_processor.preprocess(image, height=height, width=width).to(device, dtype=torch.float32) |
Problem:
The I2V pipelines document PipelineImageInput, but reject a list of PIL images. The diffusion-forcing I2V pipeline also accepts image_embeds without image, then unconditionally preprocesses image.
Impact:
Documented batched image inputs fail early, while image_embeds-only calls pass validation and fail later.
Reproduction:
importtorchfromPILimportImagefromdiffusersimportSkyReelsV2ImageToVideoPipeline, SkyReelsV2DiffusionForcingImageToVideoPipelinepipe=SkyReelsV2ImageToVideoPipeline.__new__(SkyReelsV2ImageToVideoPipeline)
pipe._callback_tensor_inputs= ["latents", "prompt_embeds", "negative_prompt_embeds"]
try:
pipe.check_inputs("a", None, [Image.new("RGB", (16, 16))], 16, 16, callback_on_step_end_tensor_inputs=["latents"])
exceptExceptionase:
print(type(e).__name__, e)
df_pipe=SkyReelsV2DiffusionForcingImageToVideoPipeline.__new__(SkyReelsV2DiffusionForcingImageToVideoPipeline)
df_pipe._callback_tensor_inputs=pipe._callback_tensor_inputsdf_pipe.check_inputs("a", None, None, 16, 16, image_embeds=torch.zeros(1, 1, 1), num_frames=9, base_num_frames=97)
print("image_embeds-only accepted")Relevant precedent:
PipelineImageInput normally includes PIL images, tensors, and lists of them.
Suggested fix:
Allow list inputs in validation, and either remove image_embeds from the diffusion-forcing I2V public API or implement a real precomputed-conditioning path.
Issue 5: Optional ftfy is still called unguarded
Affected code:
| defbasic_clean(text): |
| text=ftfy.fix_text(text) |
| text=html.unescape(html.unescape(text)) |
| defbasic_clean(text): |
| text=ftfy.fix_text(text) |
| text=html.unescape(html.unescape(text)) |
| returntext.strip() |
Problem:
ftfy is imported only when available, but basic_clean() calls ftfy.fix_text() unconditionally.
Impact:
Prompt encoding crashes in environments where optional ftfy is not installed. This is already tracked in #13112 and PR #13113.
Reproduction:
importdiffusers.pipelines.skyreels_v2.pipeline_skyreels_v2asmm.__dict__.pop("ftfy", None)
m.prompt_clean("hello")Relevant precedent:
| defbasic_clean(text): |
| ifis_ftfy_available(): |
| text=ftfy.fix_text(text) |
| text=html.unescape(html.unescape(text)) |
| returntext.strip() |
Suggested fix:
defbasic_clean(text):
ifis_ftfy_available():
text=ftfy.fix_text(text)
text=html.unescape(html.unescape(text))
returntext.strip()
Issue 6: Test coverage gaps hide SkyReels-V2 regressions
Affected code:
| classSkyReelsV2DiffusionForcingImageToVideoPipelineFastTests(PipelineTesterMixin, unittest.TestCase): |
| pipeline_class=SkyReelsV2DiffusionForcingImageToVideoPipeline |
| params=TEXT_TO_IMAGE_PARAMS- {"cross_attention_kwargs", "height", "width"} |
| batch_params=TEXT_TO_IMAGE_BATCH_PARAMS |
| image_params=TEXT_TO_IMAGE_IMAGE_PARAMS |
| image_latents_params=TEXT_TO_IMAGE_IMAGE_PARAMS |
| required_optional_params=frozenset( |
| [ |
| "num_inference_steps", |
| "generator", |
| "latents", |
| "return_dict", |
| "callback_on_step_end", |
| "callback_on_step_end_tensor_inputs", |
| ] |
| ) |
| test_xformers_attention=False |
| supports_dduf=False |
| |
| defget_dummy_components(self): |
| torch.manual_seed(0) |
| vae=AutoencoderKLWan( |
| base_dim=3, |
| z_dim=16, |
| dim_mult=[1, 1, 1, 1], |
| num_res_blocks=1, |
| temperal_downsample=[False, True, True], |
| ) |
| |
| torch.manual_seed(0) |
| scheduler=UniPCMultistepScheduler(flow_shift=5.0, use_flow_sigmas=True) |
| text_encoder=T5EncoderModel.from_pretrained("hf-internal-testing/tiny-random-t5") |
| tokenizer=AutoTokenizer.from_pretrained("hf-internal-testing/tiny-random-t5") |
| |
| torch.manual_seed(0) |
| transformer=SkyReelsV2Transformer3DModel( |
| patch_size=(1, 2, 2), |
| num_attention_heads=2, |
| attention_head_dim=12, |
| in_channels=16, |
| out_channels=16, |
| text_dim=32, |
| freq_dim=256, |
| ffn_dim=32, |
| num_layers=2, |
| cross_attn_norm=True, |
| qk_norm="rms_norm_across_heads", |
| rope_max_seq_len=32, |
| image_dim=4, |
| ) |
| |
| components= { |
| "transformer": transformer, |
| "vae": vae, |
| "scheduler": scheduler, |
| "text_encoder": text_encoder, |
| "tokenizer": tokenizer, |
| } |
| returncomponents |
| |
| defget_dummy_inputs(self, device, seed=0): |
| ifstr(device).startswith("mps"): |
| generator=torch.manual_seed(seed) |
| else: |
| generator=torch.Generator(device=device).manual_seed(seed) |
| image_height=16 |
| image_width=16 |
| image=Image.new("RGB", (image_width, image_height)) |
| inputs= { |
| "image": image, |
| "prompt": "dance monkey", |
| "negative_prompt": "negative", # TODO |
| "height": image_height, |
| "width": image_width, |
| "generator": generator, |
| "num_inference_steps": 2, |
| "guidance_scale": 5.0, |
| "num_frames": 9, |
| "max_sequence_length": 16, |
| "output_type": "pt", |
| } |
| returninputs |
| |
| deftest_inference(self): |
| device="cpu" |
| |
| components=self.get_dummy_components() |
| pipe=self.pipeline_class(**components) |
| pipe.to(device) |
| pipe.set_progress_bar_config(disable=None) |
| |
| inputs=self.get_dummy_inputs(device) |
| video=pipe(**inputs).frames |
| generated_video=video[0] |
| |
| self.assertEqual(generated_video.shape, (9, 3, 16, 16)) |
| expected_video=torch.randn(9, 3, 16, 16) |
| max_diff=np.abs(generated_video-expected_video).max() |
| self.assertLessEqual(max_diff, 1e10) |
| |
| @unittest.skip("Test not supported") |
| deftest_attention_slicing_forward_pass(self): |
| pass |
| |
| @unittest.skip("TODO: revisit failing as it requires a very high threshold to pass") |
| deftest_inference_batch_single_identical(self): |
| pass |
| |
| |
| classSkyReelsV2DiffusionForcingImageToVideoPipelineFastTests(SkyReelsV2DiffusionForcingImageToVideoPipelineFastTests): |
| classSkyReelsV2PipelineFastTests(PipelineTesterMixin, unittest.TestCase): |
| pipeline_class=SkyReelsV2Pipeline |
| params=TEXT_TO_IMAGE_PARAMS- {"cross_attention_kwargs"} |
| batch_params=TEXT_TO_IMAGE_BATCH_PARAMS |
| image_params=TEXT_TO_IMAGE_IMAGE_PARAMS |
| image_latents_params=TEXT_TO_IMAGE_IMAGE_PARAMS |
| required_optional_params=frozenset( |
| [ |
| "num_inference_steps", |
| "generator", |
| "latents", |
| "return_dict", |
| "callback_on_step_end", |
| "callback_on_step_end_tensor_inputs", |
| ] |
| ) |
| test_xformers_attention=False |
| supports_dduf=False |
| |
| defget_dummy_components(self): |
| torch.manual_seed(0) |
| vae=AutoencoderKLWan( |
| base_dim=3, |
| z_dim=16, |
| dim_mult=[1, 1, 1, 1], |
| num_res_blocks=1, |
| temperal_downsample=[False, True, True], |
| ) |
| |
| torch.manual_seed(0) |
| scheduler=UniPCMultistepScheduler(flow_shift=8.0, use_flow_sigmas=True) |
| text_encoder=T5EncoderModel.from_pretrained("hf-internal-testing/tiny-random-t5") |
| tokenizer=AutoTokenizer.from_pretrained("hf-internal-testing/tiny-random-t5") |
| |
| torch.manual_seed(0) |
| transformer=SkyReelsV2Transformer3DModel( |
| patch_size=(1, 2, 2), |
| num_attention_heads=2, |
| attention_head_dim=12, |
| in_channels=16, |
| out_channels=16, |
| text_dim=32, |
| freq_dim=256, |
| ffn_dim=32, |
| num_layers=2, |
| cross_attn_norm=True, |
| qk_norm="rms_norm_across_heads", |
| rope_max_seq_len=32, |
| ) |
| |
| components= { |
| "transformer": transformer, |
| "vae": vae, |
| "scheduler": scheduler, |
| "text_encoder": text_encoder, |
| "tokenizer": tokenizer, |
| } |
| returncomponents |
| |
| defget_dummy_inputs(self, device, seed=0): |
| ifstr(device).startswith("mps"): |
| generator=torch.manual_seed(seed) |
| else: |
| generator=torch.Generator(device=device).manual_seed(seed) |
| inputs= { |
| "prompt": "dance monkey", |
| "negative_prompt": "negative", # TODO |
| "generator": generator, |
| "num_inference_steps": 2, |
| "guidance_scale": 6.0, |
| "height": 16, |
| "width": 16, |
| "num_frames": 9, |
| "max_sequence_length": 16, |
| "output_type": "pt", |
| } |
| returninputs |
| |
| deftest_inference(self): |
| classSkyReelsV2DiffusionForcingVideoToVideoPipelineFastTests(PipelineTesterMixin, unittest.TestCase): |
| pipeline_class=SkyReelsV2DiffusionForcingVideoToVideoPipeline |
| params=TEXT_TO_IMAGE_PARAMS- {"cross_attention_kwargs"} |
| batch_params=frozenset(["video", "prompt", "negative_prompt"]) |
| image_latents_params=TEXT_TO_IMAGE_IMAGE_PARAMS |
| required_optional_params=frozenset( |
| [ |
| "num_inference_steps", |
| "generator", |
| "latents", |
| "return_dict", |
| "callback_on_step_end", |
| "callback_on_step_end_tensor_inputs", |
| ] |
| ) |
| test_xformers_attention=False |
| supports_dduf=False |
| |
| defget_dummy_components(self): |
| torch.manual_seed(0) |
| vae=AutoencoderKLWan( |
| base_dim=3, |
| z_dim=16, |
| dim_mult=[1, 1, 1, 1], |
| num_res_blocks=1, |
| temperal_downsample=[False, True, True], |
| ) |
| |
| torch.manual_seed(0) |
| scheduler=UniPCMultistepScheduler(flow_shift=5.0, use_flow_sigmas=True) |
| text_encoder=T5EncoderModel.from_pretrained("hf-internal-testing/tiny-random-t5") |
| tokenizer=AutoTokenizer.from_pretrained("hf-internal-testing/tiny-random-t5") |
| |
| torch.manual_seed(0) |
| transformer=SkyReelsV2Transformer3DModel( |
| patch_size=(1, 2, 2), |
| num_attention_heads=2, |
| attention_head_dim=12, |
| in_channels=16, |
| out_channels=16, |
| text_dim=32, |
| freq_dim=256, |
| ffn_dim=32, |
| num_layers=2, |
| cross_attn_norm=True, |
| qk_norm="rms_norm_across_heads", |
| rope_max_seq_len=32, |
| ) |
| |
| components= { |
| "transformer": transformer, |
| "vae": vae, |
| "scheduler": scheduler, |
| "text_encoder": text_encoder, |
| "tokenizer": tokenizer, |
| } |
| returncomponents |
| |
| defget_dummy_inputs(self, device, seed=0): |
| ifstr(device).startswith("mps"): |
| generator=torch.manual_seed(seed) |
| else: |
| generator=torch.Generator(device=device).manual_seed(seed) |
| |
| video= [Image.new("RGB", (16, 16))] *7 |
| inputs= { |
| "video": video, |
| "prompt": "dance monkey", |
| "negative_prompt": "negative", # TODO |
| "generator": generator, |
| "num_inference_steps": 4, |
| "guidance_scale": 6.0, |
| "height": 16, |
| "width": 16, |
| "max_sequence_length": 16, |
| "output_type": "pt", |
| "overlap_history": 3, |
| "num_frames": 17, |
| "base_num_frames": 5, |
| } |
| returninputs |
Problem:
test_skyreels_v2_df_image_to_video.py defines SkyReelsV2DiffusionForcingImageToVideoPipelineFastTests twice, so unittest discovery only exposes the second class name. The target family also has no @slow tests, and the fast tests mostly assert shapes with max_diff <= 1e10.
Impact:
The image-only DF I2V fixture is shadowed, official checkpoint behavior is untested, and regressions in output quality/scheduling can pass.
Reproduction:
frompathlibimportPathimportastfiles=list(Path("tests/pipelines/skyreels_v2").glob("test_*.py"))
print({p.name: "@slow"inp.read_text() forpinfiles})
p=Path("tests/pipelines/skyreels_v2/test_skyreels_v2_df_image_to_video.py")
classes= [n.nameforninast.parse(p.read_text()).bodyifisinstance(n, ast.ClassDef)]
print(classes)Relevant precedent:
Slow pipeline tests usually load a real small/official checkpoint path and assert deterministic output slices, not only output shapes.
Suggested fix:
Rename the second DF I2V test class, add slow tests for each public pipeline variant, and replace 1e10 thresholds with deterministic tensor slices or small visual/numeric checks.
Issue 7: Docs reference a non-existent 1.3B 720P DF checkpoint
Affected code:
| model_id ="Skywork/SkyReels-V2-DF-1.3B-720P-Diffusers" |
| vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32) |
| pipeline = SkyReelsV2DiffusionForcingImageToVideoPipeline.from_pretrained( |
| model_id, vae=vae, torch_dtype=torch.bfloat16 |
| model_id ="Skywork/SkyReels-V2-DF-1.3B-720P-Diffusers" |
| vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32) |
| pipeline = SkyReelsV2DiffusionForcingVideoToVideoPipeline.from_pretrained( |
| model_id, vae=vae, torch_dtype=torch.bfloat16 |
Problem:
The FLF2V and V2V examples use Skywork/SkyReels-V2-DF-1.3B-720P-Diffusers, but that repo does not exist. The supported list names Skywork/SkyReels-V2-DF-1.3B-540P-Diffusers and Skywork/SkyReels-V2-DF-14B-720P-Diffusers.
Impact:
Users copying the docs get a 404 before reaching pipeline execution.
Reproduction:
fromhuggingface_hubimportHfApiapi=HfApi()
api.model_info("Skywork/SkyReels-V2-DF-1.3B-720P-Diffusers")Relevant precedent:
| The following SkyReels-V2 models are supported in Diffusers: |
| -[SkyReels-V2 DF 1.3B - 540P](https://huggingface.co/Skywork/SkyReels-V2-DF-1.3B-540P-Diffusers) |
| -[SkyReels-V2 DF 14B - 540P](https://huggingface.co/Skywork/SkyReels-V2-DF-14B-540P-Diffusers) |
| -[SkyReels-V2 DF 14B - 720P](https://huggingface.co/Skywork/SkyReels-V2-DF-14B-720P-Diffusers) |
| -[SkyReels-V2 T2V 14B - 540P](https://huggingface.co/Skywork/SkyReels-V2-T2V-14B-540P-Diffusers) |
| -[SkyReels-V2 T2V 14B - 720P](https://huggingface.co/Skywork/SkyReels-V2-T2V-14B-720P-Diffusers) |
| -[SkyReels-V2 I2V 1.3B - 540P](https://huggingface.co/Skywork/SkyReels-V2-I2V-1.3B-540P-Diffusers) |
| -[SkyReels-V2 I2V 14B - 540P](https://huggingface.co/Skywork/SkyReels-V2-I2V-14B-540P-Diffusers) |
| -[SkyReels-V2 I2V 14B - 720P](https://huggingface.co/Skywork/SkyReels-V2-I2V-14B-720P-Diffusers) |
Suggested fix:
Use Skywork/SkyReels-V2-DF-14B-720P-Diffusers for 720P examples, or change the example dimensions/base frames to the existing 1.3B 540P checkpoint.
skyreels_v2model/pipeline reviewCommit tested:
0f1abc4ae8b0eb2a3b40e82a310507281144c423Review performed against the repository review rules. Reviewed public imports/lazy loading, transformer config/attention/runtime paths, all SkyReels-V2 pipelines, docs, examples, and tests. Duplicate searches found an existing duplicate only for the optional
ftfyissue: #13112 and PR #13113.Top-level imports passed in
.venv. Full pytest collection could not run because this.venvtorch build lackstorch._C._distributed_c10d, which the shared test mixins import through FSDP.Issue 1: Diffusion-forcing schedules drop tail latent frames
Affected code:
diffusers/src/diffusers/pipelines/skyreels_v2/pipeline_skyreels_v2_diffusion_forcing.py
Lines 477 to 480 in 0f1abc4
diffusers/src/diffusers/pipelines/skyreels_v2/pipeline_skyreels_v2_diffusion_forcing.py
Lines 563 to 569 in 0f1abc4
diffusers/src/diffusers/models/transformers/transformer_skyreels_v2.py
Lines 657 to 671 in 0f1abc4
Problem:
generate_timestep_matrix()floorsnum_latent_frames // causal_block_sizeand later expands back to full frames. If the latent frame count is not divisible by the block size, the remainder is omitted. The documented 720P settingbase_num_frames=121gives 31 latent frames with temporal scale 4, andcausal_block_size=5covers only 30.Impact:
The final latent frame can remain at initial noise or be skipped by the update schedule. Direct transformer calls with
num_frame_per_block > 1also build masks assuming exact divisibility.Reproduction:
Relevant precedent:
The docs recommend the 121-frame 720P setting with causal block sizing here:
diffusers/docs/source/en/api/pipelines/skyreels_v2.md
Lines 193 to 197 in 0f1abc4
Suggested fix:
Use ceil block counts and crop expanded tensors back to
num_latent_frames. In the model mask, derive block IDs per actual frame instead of repeat-interleaving a floored block count:Issue 2: Overlap noise uses global indices on local windows
Affected code:
diffusers/src/diffusers/pipelines/skyreels_v2/pipeline_skyreels_v2_diffusion_forcing.py
Lines 875 to 885 in 0f1abc4
diffusers/src/diffusers/pipelines/skyreels_v2/pipeline_skyreels_v2_diffusion_forcing_i2v.py
Lines 954 to 964 in 0f1abc4
diffusers/src/diffusers/pipelines/skyreels_v2/pipeline_skyreels_v2_diffusion_forcing_v2v.py
Lines 962 to 972 in 0f1abc4
Problem:
latent_model_inputis already sliced tovalid_interval_start:valid_interval_end, but the overlap-noise branch slices it again with global indices. It also usestorch.randn_like()directly, bypassing the user-providedgenerator.Impact:
Long-video overlap conditioning is noised on the wrong local frames, and
addnoise_condition > 0is not reproducible from the pipeline generator.Reproduction:
Relevant precedent:
Other diffusers latent preparation uses
randn_tensor(..., generator=generator)for all user-visible randomness.Suggested fix:
Issue 3:
causal_block_sizemutates serialized transformer configAffected code:
diffusers/src/diffusers/pipelines/skyreels_v2/pipeline_skyreels_v2_diffusion_forcing.py
Lines 781 to 784 in 0f1abc4
diffusers/src/diffusers/models/transformers/transformer_skyreels_v2.py
Lines 765 to 766 in 0f1abc4
Problem:
A per-call pipeline option calls
self.transformer._set_ar_attention(causal_block_size), and_set_ar_attention()writes into the model config viaregister_to_config.Impact:
One async call changes later calls that omit
causal_block_size, andsave_pretrained()will persist that call-time setting.Reproduction:
Relevant precedent:
Pipeline call arguments should not silently rewrite loadable model config.
Suggested fix:
Pass the block size as a runtime transformer forward argument, or temporarily restore the original config in a
try/finallyaround the denoising loop.Issue 4: I2V input validation rejects valid image batches and accepts unusable embeds
Affected code:
diffusers/src/diffusers/pipelines/skyreels_v2/pipeline_skyreels_v2_i2v.py
Lines 340 to 341 in 0f1abc4
diffusers/src/diffusers/pipelines/skyreels_v2/pipeline_skyreels_v2_diffusion_forcing_i2v.py
Lines 319 to 329 in 0f1abc4
diffusers/src/diffusers/pipelines/skyreels_v2/pipeline_skyreels_v2_diffusion_forcing_i2v.py
Line 872 in 0f1abc4
Problem:
The I2V pipelines document
PipelineImageInput, but reject a list of PIL images. The diffusion-forcing I2V pipeline also acceptsimage_embedswithoutimage, then unconditionally preprocessesimage.Impact:
Documented batched image inputs fail early, while
image_embeds-only calls pass validation and fail later.Reproduction:
Relevant precedent:
PipelineImageInputnormally includes PIL images, tensors, and lists of them.Suggested fix:
Allow list inputs in validation, and either remove
image_embedsfrom the diffusion-forcing I2V public API or implement a real precomputed-conditioning path.Issue 5: Optional
ftfyis still called unguardedAffected code:
diffusers/src/diffusers/pipelines/skyreels_v2/pipeline_skyreels_v2.py
Lines 90 to 92 in 0f1abc4
diffusers/src/diffusers/pipelines/skyreels_v2/pipeline_skyreels_v2_diffusion_forcing.py
Lines 97 to 100 in 0f1abc4
Problem:
ftfyis imported only when available, butbasic_clean()callsftfy.fix_text()unconditionally.Impact:
Prompt encoding crashes in environments where optional
ftfyis not installed. This is already tracked in #13112 and PR #13113.Reproduction:
Relevant precedent:
diffusers/src/diffusers/pipelines/wan/pipeline_wan.py
Lines 78 to 82 in 0f1abc4
Suggested fix:
Issue 6: Test coverage gaps hide SkyReels-V2 regressions
Affected code:
diffusers/tests/pipelines/skyreels_v2/test_skyreels_v2_df_image_to_video.py
Lines 37 to 146 in 0f1abc4
diffusers/tests/pipelines/skyreels_v2/test_skyreels_v2.py
Lines 31 to 109 in 0f1abc4
diffusers/tests/pipelines/skyreels_v2/test_skyreels_v2_df_video_to_video.py
Lines 38 to 118 in 0f1abc4
Problem:
test_skyreels_v2_df_image_to_video.pydefinesSkyReelsV2DiffusionForcingImageToVideoPipelineFastTeststwice, so unittest discovery only exposes the second class name. The target family also has no@slowtests, and the fast tests mostly assert shapes withmax_diff <= 1e10.Impact:
The image-only DF I2V fixture is shadowed, official checkpoint behavior is untested, and regressions in output quality/scheduling can pass.
Reproduction:
Relevant precedent:
Slow pipeline tests usually load a real small/official checkpoint path and assert deterministic output slices, not only output shapes.
Suggested fix:
Rename the second DF I2V test class, add slow tests for each public pipeline variant, and replace
1e10thresholds with deterministic tensor slices or small visual/numeric checks.Issue 7: Docs reference a non-existent 1.3B 720P DF checkpoint
Affected code:
diffusers/docs/source/en/api/pipelines/skyreels_v2.md
Lines 221 to 224 in 0f1abc4
diffusers/docs/source/en/api/pipelines/skyreels_v2.md
Lines 284 to 287 in 0f1abc4
Problem:
The FLF2V and V2V examples use
Skywork/SkyReels-V2-DF-1.3B-720P-Diffusers, but that repo does not exist. The supported list namesSkywork/SkyReels-V2-DF-1.3B-540P-DiffusersandSkywork/SkyReels-V2-DF-14B-720P-Diffusers.Impact:
Users copying the docs get a 404 before reaching pipeline execution.
Reproduction:
Relevant precedent:
diffusers/docs/source/en/api/pipelines/skyreels_v2.md
Lines 31 to 39 in 0f1abc4
Suggested fix:
Use
Skywork/SkyReels-V2-DF-14B-720P-Diffusersfor 720P examples, or change the example dimensions/base frames to the existing 1.3B 540P checkpoint.