Skip to content

The Modular Diffusers - #9672

Merged
yiyixuxu merged 188 commits into
mainfrom
modular-diffusers
Jul 10, 2025
Merged

The Modular Diffusers#9672
yiyixuxu merged 188 commits into
mainfrom
modular-diffusers

Conversation

@yiyixuxu

@yiyixuxuyiyixuxu commented Oct 14, 2024•
edited
Loading

Copy link
Copy Markdown
Collaborator

TO-DOs before merge

  • PAG needs to be a guider on its own
  • One more guide on how to write a PipelineBlock; and how to assemble SequentialPipelineBlocks and AutoPipelineBlocks

TO-DOs

  • guider needs to be a ConfigMixin
  • ModularPipelineBlocks.from_pretrained needs to work with official blocks
  • API docs
  • a simple test case
  • Merge the concept of ModularLoader into ModularPipeline
  • Flux/Wan/ITX
  • We overrode changes in this PR https://github.com/huggingface/diffusers/pull/11652/files, need to make sure the updated code works for this case

Documentations

  • Documentations are live here

@HuggingFaceDocBuilderDev

HuggingFaceDocBuilderDev commented Oct 14, 2024

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@yoland68

yoland68 commented Oct 16, 2024

Copy link
Copy Markdown

Very cool!

@a-r-r-o-wa-r-r-o-w mentioned this pull request Oct 25, 2024
Merged
6 tasks
@oozzy77

oozzy77 commented Oct 30, 2024•
edited
Loading

Copy link
Copy Markdown

hi this is very interesting! I'm making a Python pipeline flow visual scripting tool, that can auto-convert functions to visual nodes for fast and modular UI blocks demo. Itself is a pip package: https://pypi.org/project/nozyio/

I wanted to integrate diffusers with my flow nodes UI project but found its not very modular. But this PR may change that! Looking forward to see how this evolves.

github: https://github.com/oozzy77/nozyio happy to connect!

@yiyixuxu

yiyixuxu commented Oct 30, 2024

Copy link
Copy Markdown
CollaboratorAuthor

@oozzy77 thanks!
do you want to join a slack channel with me? if you want to experiment building something with this PR I'm eager to hear your feedback and iterate base on that

@oozzy77

oozzy77 commented Oct 31, 2024 via email

Copy link
Copy Markdown

@yiyixuxu

yiyixuxu commented Oct 31, 2024

Copy link
Copy Markdown
CollaboratorAuthor

@oozzy77 I sent an invite!

@yiyixuxuyiyixuxu mentioned this pull request Nov 6, 2024
Closed
@asomozaasomoza mentioned this pull request Nov 30, 2024
Open
2 tasks
@yiyixuxuyiyixuxu added the roadmap Add to current release roadmap label Dec 4, 2024
@hlkyhlky mentioned this pull request Dec 5, 2024
Open
a-r-r-o-w
a-r-r-o-w approved these changes Jul 7, 2025
View reviewed changes

@a-r-r-o-wa-r-r-o-w left a comment •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @yiyixuxu! The other review comments address most of the PR already. To me, I already was exposed to a full implementation early-on so it's hard for me to review as user perspective, but the docs are great and give a wide coverage of everything.

Comment threaddocs/source/en/modular_diffusers/getting_started.md Outdated
Comment threaddocs/source/en/modular_diffusers/getting_started.md Outdated
Comment threaddocs/source/en/modular_diffusers/getting_started.md
Comment threaddocs/source/en/modular_diffusers/getting_started.md Outdated
Comment threaddocs/source/en/modular_diffusers/getting_started.md Outdated
Comment threaddocs/source/en/modular_diffusers/getting_started.md Outdated
Comment threaddocs/source/en/modular_diffusers/getting_started.md Outdated
a-r-r-o-w
a-r-r-o-w reviewed Jul 7, 2025
View reviewed changes
Comment threadsrc/diffusers/hooks/_helpers.py
yiyixuxuand others added 16 commits July 8, 2025 06:15
@yiyixuxu
components manager: use shorter ID, display id instead of name
863c7df
@yiyixuxu
add a guide on components manager
a2da000
@a-r-r-o-w
Copied-from implementation of PAG-guider (#11882)
be5e10a
* update
* fix
@yiyixuxu
Merge branch 'modular-diffusers' of github.com:huggingface/diffusers …
04171c7
…into modular-diffusers
@yiyixuxu@a-r-r-o-w
Apply suggestions from code review
e6ffde2
Co-authored-by: Aryan <aryan@huggingface.co>
@yiyixuxu
update remove duplicated config for pag, and remove the description o…
5f3ebef
…f all the guiders
@yiyixuxu
add link to components manager doc
59abd95
@yiyixuxu
addreess more review comments
f95c320
@yiyixuxu
Merge branch 'main' into modular-diffusers
79166dc
@yiyixuxu
add experimental marks to all modular docs
cb9dca5
@yiyixuxu
add more docstrings + experimental marks
d27b654
@yiyixuxu
style
595581d
@yiyixuxu
Merge branch 'modular-diffusers' of github.com:huggingface/diffusers …
73c5fe8
…into modular-diffusers
@a-r-r-o-w@DN6
Merge modular diffusers with main (#11893)
de7cdf6
* [CI] Fix big GPU test marker (#11786)
* update
* update
* First Block Cache (#11180)
* update
* modify flux single blocks to make compatible with cache techniques (without too much model-specific intrusion code)
* remove debug logs
* update
* cache context for different batches of data
* fix hs residual bug for single return outputs; support ltx
* fix controlnet flux
* support flux, ltx i2v, ltx condition
* update
* update
* Update docs/source/en/api/cache.md
* Update src/diffusers/hooks/hooks.py
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
* address review comments pt. 1
* address review comments pt. 2
* cache context refacotr; address review pt. 3
* address review comments
* metadata registration with decorators instead of centralized
* support cogvideox
* support mochi
* fix
* remove unused function
* remove central registry based on review
* update
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
* fix
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
@yiyixuxu
big doc updategit status!
a935bea
@yiyixuxu
Merge branch 'modular-diffusers' of github.com:huggingface/diffusers …
2b006a2
…into modular-diffusers
yiyixuxu
yiyixuxu commented Jul 9, 2025
View reviewed changes
Comment thread.github/workflows/nightly_tests.yml
@yiyixuxu
yiyixuxu merged commit f33b89b into mainJul 10, 2025
@github-project-automationgithub-project-automationBot moved this from In Progress to Done in Diffusers RoadmapJul 10, 2025
@a-r-r-o-w
a-r-r-o-w deleted the modular-diffusers branch July 10, 2025 09:36
@vladmandicvladmandic mentioned this pull request Jul 12, 2025
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@DN6DN6DN6 approved these changes
@sayakpaulsayakpaulsayakpaul approved these changes
@stevhliustevhliuAwaiting requested review from stevhliu
@asomozaasomozaAwaiting requested review from asomoza
+3 more reviewers
@lordsoffallenlordsoffallenlordsoffallen left review comments
@a-r-r-o-wa-r-r-o-wa-r-r-o-w approved these changes
@hlkyhlkyhlky left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

roadmapAdd to current release roadmap

Projects

Diffusers Roadmap
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

13 participants

@yiyixuxu@HuggingFaceDocBuilderDev@yoland68@oozzy77@hlky@lordsoffallen@a-r-r-o-w@sayakpaul@asomoza@DN6@elismasilva@exdysa@stevhliu