After using .unload_ip_adapter() for StableDiffusionControlNetInpaintPipeline, it returns an error.
inpaint = StableDiffusionControlNetInpaintPipeline.from_pretrained(
"runwayml/stable-diffusion-inpainting",
controlnet = controlnet,
variant = "fp16",
vae = vae,
torch_dtype = torch.float16,
load_safety_checker = False
).to("cuda")
# IP adapter
image_encoder = CLIPVisionModelWithProjection.from_pretrained(
"h94/IP-Adapter",
subfolder="models/image_encoder",
torch_dtype=torch.float16,
).to("cuda")
inpaint.load_ip_adapter("h94/IP-Adapter", subfolder="models", weight_name="ip-adapter_sd15.bin")
#Generate Image
inpaint.unload_ip_adapter()
prompt = "on a platform, sky background"
neg_prompt = "3d render, cartoon, doll, anime, artwork, fake, painting, illustration"
images = inpaint(
prompt=prompt+ ", product photography, UHD, 8k, F2.8, RAW Photo, ultra detailed",
negative_prompt=neg_prompt + ",<BadDream>, (<UnrealisticDream:1.2>)",
num_inference_steps=25,
image=image_blank,
mask_image=image_mask,
control_image=depth_image,
# ip_adapter_image=image_ref,
strength=1.0,
controlnet_conditioning_scale=0.9,
num_images_per_prompt=4,
# clip_skip=1,
)
idx = 0
for each in images.images:
each.save('/content/semi-final.jpg')
image_product = Image.open(path)
image_product = scale(image_product,scale_factor)
each.paste(image_product, (x,y), image_product)
each.save(f'/content/final{idx}.jpg')
idx += 1
100%
25/25 [00:23<00:00, 1.05it/s]
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
[<ipython-input-70-0fcf450ea7c5>](https://localhost:8080/#) in <cell line: 10>()
8 neg_prompt = "3d render, cartoon, doll, anime, artwork, fake, painting, illustration"
9 ---> 10 images = inpaint(
11 prompt=prompt+ ", product photography, UHD, 8k, F2.8, RAW Photo, ultra detailed",
12 negative_prompt=neg_prompt + ",<BadDream>, (<UnrealisticDream:1.2>)",
2 frames
[/usr/local/lib/python3.10/dist-packages/diffusers/pipelines/controlnet/pipeline_controlnet_inpaint.py](https://localhost:8080/#) in run_safety_checker(self, image, device, dtype)
663 else:
664 feature_extractor_input = self.image_processor.numpy_to_pil(image)
--> 665 safety_checker_input = self.feature_extractor(feature_extractor_input, return_tensors="pt").to(device)
666 image, has_nsfw_concept = self.safety_checker(
667 images=image, clip_input=safety_checker_input.pixel_values.to(dtype)
TypeError: 'NoneType' object is not callable
### System Info
- `diffusers` version: 0.26.3
- Platform: Linux-6.1.58+-x86_64-with-glibc2.35
- Python version: 3.10.12
- PyTorch version (GPU?): 2.1.0+cu121 (True)
- Huggingface_hub version: 0.20.3
- Transformers version: 4.38.1
- Accelerate version: 0.27.2
- xFormers version: not installed
- Using GPU in script?: Yes, T4
- Using distributed or parallel set-up in script?: No
### Who can help?
@yiyixuxu @DN6
Describe the bug
After using .unload_ip_adapter() for StableDiffusionControlNetInpaintPipeline, it returns an error.
Reproduction
Logs