[Core] Refactor IPAdapterPlusImageProjection a bit

[sayakpaul]

What does this PR do?

Refer to #7988, thanks to @fabiorigano for the suggestion. Even though Fabio mentioned working on a PR for this, I had some free-time and thought I would do it. I hope that's okay with you, @fabiorigano?

In any case, would really appreciate your reviews here and if you could test this if you find some time, that would be of great help.

@rootonchair feel free to give it a look as well.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[fabiorigano]

@sayakpaul no problem, it looks really good and neat to me now!
I will do some testing as soon as I can, but maybe

diffusers/tests/models/unets/test_models_unet_2d_condition.py

Line 114 in e1df77e

defcreate_ip_adapter_plus_state_dict(model):

should be updated too, right?

[fabiorigano]

@sayakpaul I tested both simple inference and the test_ip_adapter_plus function in the test suite with the command python -m pytest -s -v tests/models -k test_ip_adapter_plus. Everything works as expected, even the test because some keys are not updated and the loading process doesn't fail. The following conditions are never true because of the new architecture of the class:

diffusers/tests/models/unets/test_models_unet_2d_condition.py

Lines 162 to 175 in e1df77e

	elif"layers.0.0"ink:
	k=k.replace("layers.0.0", "layers.0.0.norm1")
	elif"layers.0.1"ink:
	k=k.replace("layers.0.1", "layers.0.0.norm2")
	elif"layers.1.0"ink:
	k=k.replace("layers.1.0", "layers.1.0.norm1")
	elif"layers.1.1"ink:
	k=k.replace("layers.1.1", "layers.1.0.norm2")
	elif"layers.2.0"ink:
	k=k.replace("layers.2.0", "layers.2.0.norm1")
	elif"layers.2.1"ink:
	k=k.replace("layers.2.1", "layers.2.0.norm2")
	if"norm_cross"ink:

IMO they should be either removed or updated. WDYT?

[sayakpaul]

Thanks for catching this, @fabiorigano! I just updated it in 9ebacf3. WDYT?

[fabiorigano]

Looks great! @sayakpaul

[yiyixuxu]

thanks!

[sayakpaul]

Thanks to @fabiorigano once again for the suggestion and kind help.