Uh oh!
There was an error while loading. Please reload this page.
Add PIA Model/Pipeline - #6698
Conversation
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
yiyixuxu
left a comment
There was a problem hiding this comment.
super cool:)
I left some comments and questions.
for a future PR maybe we can think about how to refactor free_init. I really don't like the free_init as a pipeline feature - It makes the pipelines hard to read, and also inconsistent with our design. I think it has a similar nature to pipelines such as attend-and-excite that are modifying the denosing loop in some way in order to enhance the generations, so essentially it should be a different pipeline. Would love to hear your thoughts and ideas:)
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
HuggingFaceDocBuilderDev
commented
Jan 25, 2024
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
DN6
commented
Jan 25, 2024
@a-r-r-o-w Could you share a snippet I can use to test with? |
For comparing PIA, I've been using this colab by camenduru. I believe the problem is not with the diffusers implementation but just usual quality problems with AnimateDiff from time to time. To get similar results from both, passing torch.manual_seed(seed)
# np.random.seed(seed) # probably not neededI suspect the reason is that the original codebase (which camenduru's colab uses) has statements like |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
| if self.free_init_enabled: | ||
| latents = self._free_init_loop( | ||
| height=height, | ||
| width=width, | ||
| num_frames=num_frames, | ||
| batch_size=batch_size, | ||
| num_videos_per_prompt=num_videos_per_prompt, | ||
| denoise_args=denoise_args, | ||
| device=device, | ||
| ) | ||
| else: | ||
| latents = self._denoise_loop(**denoise_args) |
There was a problem hiding this comment.
Let's not forgot to clean up the design once the PR is merged (cc @yiyixuxu )
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
patrickvonplaten
left a comment
There was a problem hiding this comment.
Still a couple things to improve here
patrickvonplaten
commented
Jan 31, 2024
Ok to merge! Failing test is unrelated |
* update * update * updaet * add tests and docs * clean up * add to toctree * fix copies * pr review feedback * fix copies * fix tests * update docs * update * update * update docs * update * update * update * update
* update * update * updaet * add tests and docs * clean up * add to toctree * fix copies * pr review feedback * fix copies * fix tests * update docs * update * update * update docs * update * update * update * update
What does this PR do?
Adds the Personalized Image Animator (PIA) Model/Pipeline to
diffusersTODO:
Fixes # (issue)
Before submitting
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.