Update lpw_xl pipeline to latest diffusers

[a-r-r-o-w]

What does this PR do?

Adds some of the latest diffusers changes to lpw_xl. It is probably one of the most used pipelines and keeping it up-to-date with many new diffusers features would be great.

Fixes#6406.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sayakpaul@patrickvonplaten@xhinker

[a-r-r-o-w]

I'll share some example results soon. @T145 I think freeu and qkv fused should work now. Could you test it on your end as well? Thanks

[a-r-r-o-w]

@sayakpaul A few pipelines like animatediff, pixart-alpha, etc. do not support some of the latest parameters like callback_on_step_end_tensor_inputs that have been added in favour of other deprecated parameters. Maybe some of these cosmetic changes could go into this PR as well?

[sayakpaul]

Changes to those core pipelines should be done in separate PRs.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[a-r-r-o-w]

Changes to those core pipelines should be done in separate PRs.

Ah got it.

Here's a notebook testing some of the newer additions in this PR: Colab. Everything seems to be working as expected

[T145]

Can you not just extend StableDiffusionXLPipeline rather than having to copy over all the getters and such? Then just delete any # Copied over methods.

[a-r-r-o-w]

Hi @T145, please refer to the diffusers philosophy guide. Any pipelines as such are not meant to be derived from, and all pipelines should derive from DiffusionPipeline and implement repeated methods using the # Copied from mechanism. Also, see this discussion here.

[T145]

So when are we merging this bad boy?

[T145]

While this is still open:

Traceback (most recent call last):
File "-\test.py", line 182, in <module>
generate_image(prompt, neg_prompt)
File "-\test.py", line 81, in generate_image
pipe.load_textual_inversion(state_dict["clip_g"], token, text_encoder=pipe.text_encoder_2, tokenizer=base.tokenizer_2)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "-\configuration_utils.py", line 142, in __getattr__
raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'SDXLLongPromptWeightingPipeline' object has no attribute 'load_textual_inversion'

[T145]

Traceback (most recent call last):
File "-\paint.py", line 179, in <module>
generate_image(prompt, neg_prompt, steps=30, seed=332518660889778939)
File "-\paint.py", line 134, in generate_image
image = pipe(
^^^^^
File "-\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "-\diffusers_modules\git\lpw_stable_diffusion_xl.py", line 1664, in __call__
timestep_cond = self.get_guidance_scale_embedding(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "-\diffusers\configuration_utils.py", line 142, in __getattr__
raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'SDXLLongPromptWeightingPipeline' object has no attribute 'get_guidance_scale_embedding'

[patrickvonplaten]

Thanks!