ControlNet Reference Only for SD XL

[jelling]

Is your feature request related to a problem? Please describe.

There is currently a Diffusers example for using "Reference Only" with ControlNet. But there is not yet one for XL.

Describe the solution you'd like

It would be great to have an example for SD XL.

Please and thanks!

[sayakpaul]

Feel free to add a PR :)

[jelling]

Yeah, I may do that. Before I dive in, are there any known reasons why reference only wouldn't work with the XL controlnet?

(What I'm working around is that because textual inversions aren't supported yet for XL - thank you for your help with understanding that btw - I now need a new solution for a problem that I was previously solving with TIs.)

[sayakpaul]

The ControlNet accepts additional embeddings for SDXL. Maybe that needs to be rejigged.

[charlesmccarthy]

any updates on this?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[reimager]

Commenting so this isn't stale. Still following this issue. This pipeline would be really useful.

[sayakpaul]

Welcome to PRs.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[DN6]

PRs are welcome for this addition. But I'm just curious to understand why IP adapters wouldn't be able to provide similar functionality?