Effective region mask for controlnet

[Suprhimp]

Hi, I just want to ask is there any way to use controlnet with mask like this

As you know comfyui, webui support effective region (mask for controlnet affect).
But I can't find how to do this with diffusers.

[alexblattner]

yes, multidiffusion. it depends on your SD version though. I made a custom pipeline that handles that on my profile for sd1.5

[Suprhimp]

Thanks for the interest.

But I'm looking for sdxl ;)

[Suprhimp]

#6847

I can find attention mask for ip-adapter is here. but still I can't find attention mask for controlnet (sdxl)

[asomoza]

Hi, we don't have attention mask or any kind of effective masking for controlnets. We're open if someone from the community wants to add this though.

If not, probably I will work on this with modular diffusers when I have the time.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[Suprhimp]

Can you give me a little hint about modular diffusers that you said..?

[asomoza]

Modular Diffusers it's still a work in progress in this PR but with it, you will be able to generate your own pipelines with whatever you want, so regional prompting as a whole (not just controlnets) it's ideal to demonstrate how to use modular diffusers.

[Suprhimp]

seems like a new generation for diffusers. I'll look forward to that PR be merged.