Skip to content

kandinsky2_2 model/pipeline review #13596

Description

@hlky

kandinsky2_2 model/pipeline review

Commit tested: 0f1abc4ae8b0eb2a3b40e82a310507281144c423

Review performed against the repository review rules.

Duplicate search checked kandinsky2_2, KandinskyV22, affected class/file names, and the specific failure modes. Existing related items: #4183 covers the ControlNet guidance_scale <= 1 subset of Issue 2; #4818 is related to stale PriorEmb2Emb latents plumbing, but I did not find an exact duplicate for the current interpolate() failure.

Issue 1: PriorEmb2Emb.interpolate() cannot handle documented text entries

Affected code:

forcond, weightinzip(images_and_prompts, weights):
ifisinstance(cond, str):
image_emb=self(
cond,
num_inference_steps=num_inference_steps,
num_images_per_prompt=num_images_per_prompt,
generator=generator,
latents=latents,
negative_prompt=negative_prior_prompt,
guidance_scale=guidance_scale,
).image_embeds.unsqueeze(0)

@torch.no_grad()
@replace_example_docstring(EXAMPLE_DOC_STRING)
def__call__(
self,
prompt: str|list[str],
image: torch.Tensor|list[torch.Tensor] |PIL.Image.Image|list[PIL.Image.Image],
strength: float=0.3,
negative_prompt: str|list[str] |None=None,
num_images_per_prompt: int=1,
num_inference_steps: int=25,
generator: torch.Generator|list[torch.Generator] |None=None,
guidance_scale: float=4.0,
output_type: str|None="pt", # pt only
return_dict: bool=True,
):

image_emb=torch.cat(image_embeddings).sum(dim=0)
returnKandinskyPriorPipelineOutput(image_embeds=image_emb, negative_image_embeds=torch.randn_like(image_emb))

Problem:
interpolate() advertises list[str | PIL.Image.Image | torch.Tensor], but the string branch calls self(..., latents=latents) while KandinskyV22PriorEmb2EmbPipeline.__call__ has no latents parameter and requires image. The same method also returns torch.randn_like(image_emb) for negative_image_embeds, making the negative conditioning random and unrelated to negative_prompt.

Impact:
The documented mixed text/image interpolation workflow fails immediately for text entries. Image-only interpolation is nondeterministic on the negative branch and can change decoder CFG behavior across calls.

Reproduction:

fromdiffusersimportKandinskyV22PriorEmb2EmbPipelinepipe=KandinskyV22PriorEmb2EmbPipeline(
prior=None, image_encoder=None, text_encoder=None,
tokenizer=None, scheduler=None, image_processor=None,
)
try:
pipe.interpolate(["a cat"], [1.0])
exceptTypeErrorase:
print(type(e).__name__, str(e).split("\n")[0])

Relevant precedent:
KandinskyV22PriorPipeline.interpolate() supports text entries by calling a compatible text-only prior path and derives the negative image embedding from a real negative/zero embedding:

image_embeddings= []
forcond, weightinzip(images_and_prompts, weights):
ifisinstance(cond, str):
image_emb=self(
cond,
num_inference_steps=num_inference_steps,
num_images_per_prompt=num_images_per_prompt,
generator=generator,
latents=latents,
negative_prompt=negative_prior_prompt,
guidance_scale=guidance_scale,
).image_embeds.unsqueeze(0)
elifisinstance(cond, (PIL.Image.Image, torch.Tensor)):
ifisinstance(cond, PIL.Image.Image):
cond= (
self.image_processor(cond, return_tensors="pt")
.pixel_values[0]
.unsqueeze(0)
.to(dtype=self.image_encoder.dtype, device=device)
)
image_emb=self.image_encoder(cond)["image_embeds"].repeat(num_images_per_prompt, 1).unsqueeze(0)
else:
raiseValueError(
f"`images_and_prompts` can only contains elements to be of type `str`, `PIL.Image.Image` or `torch.Tensor` but is {type(cond)}"
)
image_embeddings.append(image_emb*weight)
image_emb=torch.cat(image_embeddings).sum(dim=0)
out_zero=self(
negative_prompt,
num_inference_steps=num_inference_steps,
num_images_per_prompt=num_images_per_prompt,
generator=generator,
latents=latents,
negative_prompt=negative_prior_prompt,
guidance_scale=guidance_scale,
)
zero_image_emb=out_zero.negative_image_embedsifnegative_prompt==""elseout_zero.image_embeds
returnKandinskyPriorPipelineOutput(image_embeds=image_emb, negative_image_embeds=zero_image_emb)

Suggested fix:
Implement the text branch by reusing/copying the text-only prior logic from KandinskyV22PriorPipeline, or narrow the accepted input types and docs to image/tensor only. Replace torch.randn_like(image_emb) with a deterministic negative embedding, likely get_zero_embed(...) for the empty negative case and a real negative-prompt path when supported.

Issue 2: Decoder no-CFG paths do not repeat or cast conditioning tensors

Affected code:

ifisinstance(image_embeds, list):
image_embeds=torch.cat(image_embeds, dim=0)
batch_size=image_embeds.shape[0] *num_images_per_prompt
ifisinstance(negative_image_embeds, list):
negative_image_embeds=torch.cat(negative_image_embeds, dim=0)
ifself.do_classifier_free_guidance:
image_embeds=image_embeds.repeat_interleave(num_images_per_prompt, dim=0)
negative_image_embeds=negative_image_embeds.repeat_interleave(num_images_per_prompt, dim=0)
image_embeds=torch.cat([negative_image_embeds, image_embeds], dim=0).to(
dtype=self.unet.dtype, device=device
)

ifisinstance(image_embeds, list):
image_embeds=torch.cat(image_embeds, dim=0)
batch_size=image_embeds.shape[0]
ifisinstance(negative_image_embeds, list):
negative_image_embeds=torch.cat(negative_image_embeds, dim=0)
ifself.do_classifier_free_guidance:
image_embeds=image_embeds.repeat_interleave(num_images_per_prompt, dim=0)
negative_image_embeds=negative_image_embeds.repeat_interleave(num_images_per_prompt, dim=0)
image_embeds=torch.cat([negative_image_embeds, image_embeds], dim=0).to(
dtype=self.unet.dtype, device=device
)

ifisinstance(image_embeds, list):
image_embeds=torch.cat(image_embeds, dim=0)
batch_size=image_embeds.shape[0] *num_images_per_prompt
ifisinstance(negative_image_embeds, list):
negative_image_embeds=torch.cat(negative_image_embeds, dim=0)
ifself.do_classifier_free_guidance:
image_embeds=image_embeds.repeat_interleave(num_images_per_prompt, dim=0)
negative_image_embeds=negative_image_embeds.repeat_interleave(num_images_per_prompt, dim=0)
image_embeds=torch.cat([negative_image_embeds, image_embeds], dim=0).to(
dtype=self.unet.dtype, device=device
)

ifisinstance(image_embeds, list):
image_embeds=torch.cat(image_embeds, dim=0)
ifisinstance(negative_image_embeds, list):
negative_image_embeds=torch.cat(negative_image_embeds, dim=0)
ifisinstance(hint, list):
hint=torch.cat(hint, dim=0)
batch_size=image_embeds.shape[0] *num_images_per_prompt
ifdo_classifier_free_guidance:
image_embeds=image_embeds.repeat_interleave(num_images_per_prompt, dim=0)
negative_image_embeds=negative_image_embeds.repeat_interleave(num_images_per_prompt, dim=0)
hint=hint.repeat_interleave(num_images_per_prompt, dim=0)
image_embeds=torch.cat([negative_image_embeds, image_embeds], dim=0).to(
dtype=self.unet.dtype, device=device
)
hint=torch.cat([hint, hint], dim=0).to(dtype=self.unet.dtype, device=device)

ifisinstance(image_embeds, list):
image_embeds=torch.cat(image_embeds, dim=0)
ifisinstance(negative_image_embeds, list):
negative_image_embeds=torch.cat(negative_image_embeds, dim=0)
ifisinstance(hint, list):
hint=torch.cat(hint, dim=0)
batch_size=image_embeds.shape[0]
ifdo_classifier_free_guidance:
image_embeds=image_embeds.repeat_interleave(num_images_per_prompt, dim=0)
negative_image_embeds=negative_image_embeds.repeat_interleave(num_images_per_prompt, dim=0)
hint=hint.repeat_interleave(num_images_per_prompt, dim=0)
image_embeds=torch.cat([negative_image_embeds, image_embeds], dim=0).to(
dtype=self.unet.dtype, device=device
)
hint=torch.cat([hint, hint], dim=0).to(dtype=self.unet.dtype, device=device)

Problem:
image_embeds, and hint for ControlNet, are repeated for num_images_per_prompt and moved to self.unet.dtype/device only inside the CFG branch. With guidance_scale <= 1, latents are sized for batch * num_images_per_prompt, but conditioning remains at the original batch size and dtype/device.

Impact:
No-CFG generation can fail for num_images_per_prompt > 1, half-precision pipelines, CPU-to-accelerator inputs, and ControlNet hints. The ControlNet subset is already reported in #4183.

Reproduction:

fromtypesimportSimpleNamespaceimporttorchfromdiffusersimportKandinskyV22PipelineclassFakeUNet:
dtype=torch.float32config=SimpleNamespace(in_channels=4)
def__call__(self, sample, timestep, encoder_hidden_states=None, added_cond_kwargs=None, return_dict=False):
assertadded_cond_kwargs["image_embeds"].shape[0] ==sample.shape[0], (
added_cond_kwargs["image_embeds"].shape, sample.shape
)
return (torch.zeros(sample.shape[0], sample.shape[1] *2, sample.shape[2], sample.shape[3]),)
classFakeScheduler:
init_noise_sigma=1.0config=SimpleNamespace(variance_type="learned")
defset_timesteps(self, *args, **kwargs): self.timesteps=torch.tensor([1])
defstep(self, noise_pred, t, latents, generator=None): return (latents,)
classFakeMovq:
config=SimpleNamespace(block_out_channels=[1, 1], latent_channels=4)
pipe=KandinskyV22Pipeline(FakeUNet(), FakeScheduler(), FakeMovq())
pipe.set_progress_bar_config(disable=True)
pipe(
image_embeds=torch.randn(1, 32),
negative_image_embeds=torch.randn(1, 32),
num_images_per_prompt=2,
guidance_scale=1.0,
output_type="latent",
)

Relevant precedent:
The prior pipeline expands text conditioning before CFG concatenation:

prompt_embeds=prompt_embeds.repeat_interleave(num_images_per_prompt, dim=0)
text_encoder_hidden_states=text_encoder_hidden_states.repeat_interleave(num_images_per_prompt, dim=0)
text_mask=text_mask.repeat_interleave(num_images_per_prompt, dim=0)

Suggested fix:

image_embeds=image_embeds.repeat_interleave(num_images_per_prompt, dim=0).to(
dtype=self.unet.dtype, device=device
)
ifself.do_classifier_free_guidance:
negative_image_embeds=negative_image_embeds.repeat_interleave(num_images_per_prompt, dim=0).to(
dtype=self.unet.dtype, device=device
)
image_embeds=torch.cat([negative_image_embeds, image_embeds], dim=0)

Apply the same pattern to hint in both ControlNet pipelines before duplicating it for CFG.

Issue 3: Inpainting batches preserve the first image/mask for every sample

Affected code:

init_latents_proper=image[:1]
init_mask=mask_image[:1]
ifi<len(timesteps) -1:
noise_timestep=timesteps[i+1]
init_latents_proper=self.scheduler.add_noise(
init_latents_proper, noise, torch.tensor([noise_timestep])
)
latents=init_mask*init_latents_proper+ (1-init_mask) *latents

# post-processing
latents=mask_image[:1] *image[:1] + (1-mask_image[:1]) *latents

Problem:
The inpaint loop uses image[:1] and mask_image[:1] when reinserting preserved regions. For batched inputs, every sample uses the first encoded image and first mask instead of its own.

Impact:
Batched inpainting silently produces wrong preserved regions. This can make all batch outputs inherit the first input image under preserved mask areas.

Reproduction:

fromtypesimportSimpleNamespaceimporttorchfromdiffusersimportKandinskyV22InpaintPipelineclassFakeUNet:
dtype=torch.float32config=SimpleNamespace(in_channels=9)
def__call__(self, sample, timestep, encoder_hidden_states=None, added_cond_kwargs=None, return_dict=False):
return (torch.zeros(sample.shape[0], 8, sample.shape[2], sample.shape[3]),)
classFakeScheduler:
init_noise_sigma=1.0config=SimpleNamespace(variance_type="learned")
defset_timesteps(self, *args, **kwargs): self.timesteps=torch.tensor([1])
defstep(self, noise_pred, t, latents, generator=None): return (latents,)
classFakeMovq:
config=SimpleNamespace(block_out_channels=[1, 1], latent_channels=4)
defencode(self, image):
values=image.mean(dim=(1, 2, 3), keepdim=True)
return {"latents": values.expand(image.shape[0], 4, 32, 32)}
pipe=KandinskyV22InpaintPipeline(FakeUNet(), FakeScheduler(), FakeMovq())
pipe.set_progress_bar_config(disable=True)
image=torch.stack([torch.full((3, 64, 64), -1.0), torch.full((3, 64, 64), 1.0)])
mask=torch.zeros(2, 1, 64, 64) # preserve both inputsout=pipe(
image_embeds=torch.randn(2, 32),
negative_image_embeds=torch.randn(2, 32),
image=image,
mask_image=mask,
output_type="latent",
).imagesprint(torch.unique(out[0]).item(), torch.unique(out[1]).item())

Relevant precedent:
Stable Diffusion inpainting keeps per-sample mask and masked-image latents batched through the denoising loop rather than slicing to the first item.

Suggested fix:

init_latents=image.repeat_interleave(num_images_per_prompt, dim=0)
init_mask=mask_image.repeat_interleave(num_images_per_prompt, dim=0)
mask_image=init_maskmasked_image=init_latents*init_maskifself.do_classifier_free_guidance:
mask_image=mask_image.repeat(2, 1, 1, 1)
masked_image=masked_image.repeat(2, 1, 1, 1)
...
init_latents_proper=init_latentsifi<len(timesteps) -1:
noise_timestep=timesteps[i+1]
init_latents_proper=self.scheduler.add_noise(init_latents, noise, noise_timestep[None])
latents=init_mask*init_latents_proper+ (1-init_mask) *latents
...
latents=init_mask*init_latents+ (1-init_mask) *latents

Issue 4: User-provided latents are not cast to the requested dtype

Affected code:

# Copied from diffusers.pipelines.deprecated.unclip.pipeline_unclip.UnCLIPPipeline.prepare_latents
defprepare_latents(self, shape, dtype, device, generator, latents, scheduler):
iflatentsisNone:
latents=randn_tensor(shape, generator=generator, device=device, dtype=dtype)
else:
iflatents.shape!=shape:
raiseValueError(f"Unexpected latents shape, got {latents.shape}, expected {shape}")
latents=latents.to(device)
latents=latents*scheduler.init_noise_sigma

# Copied from diffusers.pipelines.deprecated.unclip.pipeline_unclip.UnCLIPPipeline.prepare_latents
defprepare_latents(self, shape, dtype, device, generator, latents, scheduler):
iflatentsisNone:
latents=randn_tensor(shape, generator=generator, device=device, dtype=dtype)
else:
iflatents.shape!=shape:
raiseValueError(f"Unexpected latents shape, got {latents.shape}, expected {shape}")
latents=latents.to(device)
latents=latents*scheduler.init_noise_sigma

defprepare_latents(self, shape, dtype, device, generator, latents, scheduler):
iflatentsisNone:
latents=randn_tensor(shape, generator=generator, device=device, dtype=dtype)
else:
iflatents.shape!=shape:
raiseValueError(f"Unexpected latents shape, got {latents.shape}, expected {shape}")
latents=latents.to(device)
latents=latents*scheduler.init_noise_sigma

defprepare_latents(self, shape, dtype, device, generator, latents, scheduler):
iflatentsisNone:
latents=randn_tensor(shape, generator=generator, device=device, dtype=dtype)
else:
iflatents.shape!=shape:
raiseValueError(f"Unexpected latents shape, got {latents.shape}, expected {shape}")
latents=latents.to(device)
latents=latents*scheduler.init_noise_sigma

Problem:
The copied prepare_latents() blocks cast generated latents to dtype, but user-provided latents only call .to(device). In half precision, a float32 latent tensor stays float32 and can be passed into fp16 modules.

Impact:
Supplying reusable latents to fp16 pipelines can fail with dtype mismatch errors or force unintended float32 compute.

Reproduction:

fromtypesimportSimpleNamespaceimporttorchfromdiffusersimportKandinskyV22Pipelinepipe=KandinskyV22Pipeline.__new__(KandinskyV22Pipeline)
scheduler=SimpleNamespace(init_noise_sigma=1.0)
latents=torch.randn(1, 4, 32, 32, dtype=torch.float32)
out=KandinskyV22Pipeline.prepare_latents(
pipe, latents.shape, torch.float16, torch.device("cpu"), None, latents, scheduler
)
print(out.dtype) # torch.float32, expected torch.float16

Relevant precedent:
Flux casts provided latents with both device and dtype:

iflatentsisnotNone:
latent_image_ids=self._prepare_latent_image_ids(batch_size, height//2, width//2, device, dtype)
returnlatents.to(device=device, dtype=dtype), latent_image_ids

Suggested fix:

latents=latents.to(device=device, dtype=dtype)

Because this is a copied block, either update the copied source and run make fix-copies, or remove the copy annotation if Kandinsky needs target-specific behavior.

Issue 5: Deprecated callback crashes when callback_steps is omitted

Affected code:

callback=kwargs.pop("callback", None)
callback_steps=kwargs.pop("callback_steps", None)
ifcallbackisnotNone:
deprecate(
"callback",
"1.0.0",
"Passing `callback` as an input argument to `__call__` is deprecated, consider use `callback_on_step_end`",
)
ifcallback_stepsisnotNone:
deprecate(
"callback_steps",
"1.0.0",
"Passing `callback_steps` as an input argument to `__call__` is deprecated, consider use `callback_on_step_end`",
)

ifcallbackisnotNoneandi%callback_steps==0:
step_idx=i//getattr(self.scheduler, "order", 1)
callback(step_idx, t, latents)

callback=kwargs.pop("callback", None)
callback_steps=kwargs.pop("callback_steps", None)
ifcallbackisnotNone:
deprecate(
"callback",
"1.0.0",
"Passing `callback` as an input argument to `__call__` is deprecated, consider use `callback_on_step_end`",
)
ifcallback_stepsisnotNone:
deprecate(
"callback_steps",
"1.0.0",
"Passing `callback_steps` as an input argument to `__call__` is deprecated, consider use `callback_on_step_end`",
)

callback=kwargs.pop("callback", None)
callback_steps=kwargs.pop("callback_steps", None)
ifcallbackisnotNone:
deprecate(
"callback",
"1.0.0",
"Passing `callback` as an input argument to `__call__` is deprecated, consider use `callback_on_step_end`",
)
ifcallback_stepsisnotNone:
deprecate(
"callback_steps",
"1.0.0",
"Passing `callback_steps` as an input argument to `__call__` is deprecated, consider use `callback_on_step_end`",
)

Problem:
The deprecated callback kwarg is still accepted, but callback_steps defaults to None after kwargs.pop("callback_steps", None). The loop then evaluates i % callback_steps, causing TypeError.

Impact:
Backward-compatible callback usage fails unless users also pass a deprecated callback_steps kwarg.

Reproduction:

fromtypesimportSimpleNamespaceimporttorchfromdiffusersimportKandinskyV22PipelineclassFakeUNet:
dtype=torch.float32config=SimpleNamespace(in_channels=4)
def__call__(self, sample, timestep, encoder_hidden_states=None, added_cond_kwargs=None, return_dict=False):
return (torch.zeros(sample.shape[0], 8, sample.shape[2], sample.shape[3]),)
classFakeScheduler:
init_noise_sigma=1.0config=SimpleNamespace(variance_type="learned")
defset_timesteps(self, *args, **kwargs): self.timesteps=torch.tensor([1])
defstep(self, noise_pred, t, latents, generator=None): return (latents,)
classFakeMovq:
config=SimpleNamespace(block_out_channels=[1, 1], latent_channels=4)
pipe=KandinskyV22Pipeline(FakeUNet(), FakeScheduler(), FakeMovq())
pipe.set_progress_bar_config(disable=True)
pipe(
image_embeds=torch.randn(1, 32),
negative_image_embeds=torch.randn(1, 32),
output_type="latent",
callback=lambdastep, t, latents: None,
)

Relevant precedent:
The ControlNet variants still expose callback_steps: int = 1 in the signature:

generator: torch.Generator|list[torch.Generator] |None=None,
latents: torch.Tensor|None=None,
output_type: str|None="pil",
callback: Callable[[int, int, torch.Tensor], None] |None=None,
callback_steps: int=1,
return_dict: bool=True,

Suggested fix:

callback_steps=kwargs.pop("callback_steps", 1)
ifcallback_stepsisNone:
callback_steps=1

Also validate that callback_steps is a positive integer before the denoising loop.

Issue 6: Slow coverage is missing for several exported pipelines

Affected code:

deftest_kandinsky_prior(self):
device="cpu"
components=self.get_dummy_components()
pipe=self.pipeline_class(**components)
pipe=pipe.to(device)
pipe.set_progress_bar_config(disable=None)
output=pipe(**self.get_dummy_inputs(device))
image=output.image_embeds
image_from_tuple=pipe(
**self.get_dummy_inputs(device),
return_dict=False,
)[0]
image_slice=image[0, -10:]
image_from_tuple_slice=image_from_tuple[0, -10:]
assertimage.shape== (1, 32)
expected_slice=np.array(
[-0.5948, 0.1875, -0.1523, -1.1995, -1.4061, -0.6367, -1.4607, -0.6406, 0.8793, -0.3891]
)
assertnp.abs(image_slice.flatten() -expected_slice).max() <1e-2
assertnp.abs(image_from_tuple_slice.flatten() -expected_slice).max() <1e-2
@skip_mps
deftest_inference_batch_single_identical(self):
self._test_inference_batch_single_identical(expected_max_diff=1e-3)
@skip_mps
deftest_attention_slicing_forward_pass(self):
test_max_difference=torch_device=="cpu"
test_mean_pixel_difference=False
self._test_attention_slicing_forward_pass(
test_max_difference=test_max_difference,
test_mean_pixel_difference=test_mean_pixel_difference,
)
# override default test because no output_type "latent", use "pt" instead
deftest_callback_inputs(self):

deftest_kandinsky_prior_emb2emb(self):
device="cpu"
components=self.get_dummy_components()
pipe=self.pipeline_class(**components)
pipe=pipe.to(device)
pipe.set_progress_bar_config(disable=None)
output=pipe(**self.get_dummy_inputs(device))
image=output.image_embeds
image_from_tuple=pipe(
**self.get_dummy_inputs(device),
return_dict=False,
)[0]
image_slice=image[0, -10:]
image_from_tuple_slice=image_from_tuple[0, -10:]
assertimage.shape== (1, 32)
expected_slice=np.array(
[-0.8947, 0.7225, -0.2400, -1.4224, -1.9268, -1.1454, -1.8220, -0.7972, 1.0465, -0.5207]
)
assertnp.abs(image_slice.flatten() -expected_slice).max() <1e-2
assertnp.abs(image_from_tuple_slice.flatten() -expected_slice).max() <1e-2
@skip_mps
deftest_inference_batch_single_identical(self):
self._test_inference_batch_single_identical(expected_max_diff=1e-2)
@skip_mps
deftest_attention_slicing_forward_pass(self):
test_max_difference=torch_device=="cpu"

deftest_float16_inference(self):
super().test_float16_inference(expected_max_diff=1e-1)
deftest_inference_batch_single_identical(self):
super().test_inference_batch_single_identical(expected_max_diff=5e-4)
@nightly
@require_torch_accelerator
classKandinskyV22ControlnetPipelineIntegrationTests(unittest.TestCase):
defsetUp(self):
# clean up the VRAM before each test
super().setUp()
gc.collect()
backend_empty_cache(torch_device)
deftearDown(self):
# clean up the VRAM after each test
super().tearDown()
gc.collect()
backend_empty_cache(torch_device)
deftest_kandinsky_controlnet(self):

deftest_inference_batch_single_identical(self):
super().test_inference_batch_single_identical(expected_max_diff=1.75e-3)
deftest_float16_inference(self):
super().test_float16_inference(expected_max_diff=2e-1)
@nightly
@require_torch_accelerator
classKandinskyV22ControlnetImg2ImgPipelineIntegrationTests(unittest.TestCase):
defsetUp(self):
# clean up the VRAM before each test
super().setUp()
gc.collect()
backend_empty_cache(torch_device)
deftearDown(self):
# clean up the VRAM after each test
super().tearDown()
gc.collect()
backend_empty_cache(torch_device)
deftest_kandinsky_controlnet_img2img(self):

classKandinskyV22PipelineCombinedFastTests(PipelineTesterMixin, unittest.TestCase):
pipeline_class=KandinskyV22CombinedPipeline
params= ["prompt"]
batch_params= ["prompt", "negative_prompt"]
required_optional_params= [
"generator",
"height",
"width",
"latents",
"guidance_scale",
"negative_prompt",
"num_inference_steps",
"return_dict",
"guidance_scale",
"num_images_per_prompt",
"output_type",
"return_dict",
]
test_xformers_attention=True
callback_cfg_params= ["image_embds"]
supports_dduf=False

Problem:
The family has fast tests, and text/img2img/inpaint decoder pipelines have @slow integration tests. But KandinskyV22PriorPipeline, KandinskyV22PriorEmb2EmbPipeline, all three combined pipelines, and both ControlNet pipelines do not have @slow tests. The ControlNet integration tests are @nightly, which means they are not collected by slow CI.

Impact:
Published checkpoints and connected-pipeline loading paths can regress without slow-suite coverage. This also leaves the broken PriorEmb2Emb.interpolate() path untested.

Reproduction:

frompathlibimportPathforpathinsorted(Path("tests/pipelines/kandinsky2_2").glob("test_*.py")):
text=path.read_text()
print(f"{path.name}: slow={'@slow'intext}, nightly={'@nightly'intext}")

Relevant precedent:
Existing slow coverage for decoder text/img2img/inpaint:

@slow
@require_torch_accelerator
classKandinskyV22PipelineIntegrationTests(unittest.TestCase):
defsetUp(self):
# clean up the VRAM before each test
super().setUp()
gc.collect()
backend_empty_cache(torch_device)
deftearDown(self):
# clean up the VRAM after each test
super().tearDown()
gc.collect()
backend_empty_cache(torch_device)
deftest_kandinsky_text2img(self):

@slow
@require_torch_accelerator
classKandinskyV22Img2ImgPipelineIntegrationTests(unittest.TestCase):
defsetUp(self):
# clean up the VRAM before each test
super().setUp()
gc.collect()
backend_empty_cache(torch_device)
deftearDown(self):
# clean up the VRAM after each test
super().tearDown()
gc.collect()
backend_empty_cache(torch_device)
deftest_kandinsky_img2img(self):

@slow
@require_torch_accelerator
classKandinskyV22InpaintPipelineIntegrationTests(unittest.TestCase):
defsetUp(self):
# clean up the VRAM before each test
super().setUp()
gc.collect()
backend_empty_cache(torch_device)
deftearDown(self):
# clean up the VRAM after each test
super().tearDown()
gc.collect()
backend_empty_cache(torch_device)
deftest_kandinsky_inpaint(self):

Suggested fix:
Add @slow integration tests for prior, prior emb2emb interpolation, combined text/img2img/inpaint loading, and ControlNet text/img2img. Keep @nightly if desired, but add @slow so slow CI covers the family.

Notes

Public exports, lazy loading, top-level imports, dummy objects, AutoPipeline registrations, docs, examples, and fast tests were checked. Top-level imports for all exported KandinskyV22* classes succeed locally.

I attempted .venv pytest collection, but this environment's torch build fails during test import with ModuleNotFoundError: No module named 'torch._C._distributed_c10d'; 'torch._C' is not a package, before Kandinsky tests are collected.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions