Skip to content

[Kolors] Add PAG#8934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sayakpaul merged 16 commits into from
Aug 7, 2024
Merged

[Kolors] Add PAG #8934

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
e500206
txt2img pag added
asomoza Jul 22, 2024
a4076dc
Merge branch 'main' into kolors-pag
asomoza Jul 22, 2024
8091ba8
autopipe added, fixed case
asomoza Jul 22, 2024
ef3b5d1
style
asomoza Jul 22, 2024
69784c1
Merge branch 'main' into kolors-pag
asomoza Jul 26, 2024
01b07c8
apply suggestions
asomoza Jul 26, 2024
c31fe14
added fast tests, added todo tests
asomoza Jul 26, 2024
4811fb7
Merge branch 'main' into kolors-pag
asomoza Aug 5, 2024
c4f5e87
Merge branch 'main' into kolors-pag
asomoza Aug 7, 2024
907efde
revert dummy objects for kolors
asomoza Aug 7, 2024
83af5d6
fix pag dummies
asomoza Aug 7, 2024
ba58b8d
fix test imports
asomoza Aug 7, 2024
0780551
update pag tests
asomoza Aug 7, 2024
db41a51
Merge branch 'main' into kolors-pag
asomoza Aug 7, 2024
61e08a5
Merge branch 'main' into kolors-pag
sayakpaul Aug 7, 2024
e74ce93
add kolor pag to docs
asomoza Aug 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/source/en/api/pipelines/pag.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -43,6 +43,11 @@ Since RegEx is supported as a way for matching layer identifiers, it is crucial
- all
- __call__

## KolorsPAGPipeline
[[autodoc]] KolorsPAGPipeline
- all
- __call__

## StableDiffusionPAGPipeline
[[autodoc]] StableDiffusionPAGPipeline
- all
Expand Down
6 changes: 2 additions & 4 deletions src/diffusers/__init__.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -280,8 +280,6 @@
"KandinskyV22Pipeline",
"KandinskyV22PriorEmb2EmbPipeline",
"KandinskyV22PriorPipeline",
"KolorsImg2ImgPipeline",
"KolorsPipeline",
"LatentConsistencyModelImg2ImgPipeline",
"LatentConsistencyModelPipeline",
"LattePipeline",
Expand DownExpand Up@@ -397,7 +395,7 @@
]

else:
_import_structure["pipelines"].extend(["KolorsImg2ImgPipeline", "KolorsPipeline"])
_import_structure["pipelines"].extend(["KolorsImg2ImgPipeline", "KolorsPAGPipeline", "KolorsPipeline"])

try:
if not (is_torch_available() and is_transformers_available() and is_onnx_available()):
Expand DownExpand Up@@ -820,7 +818,7 @@
except OptionalDependencyNotAvailable:
from .utils.dummy_torch_and_transformers_and_sentencepiece_objects import * # noqa F403
else:
from .pipelines import KolorsImg2ImgPipeline, KolorsPipeline
from .pipelines import KolorsImg2ImgPipeline, KolorsPAGPipeline, KolorsPipeline
try:
if not (is_torch_available() and is_transformers_available() and is_onnx_available()):
raise OptionalDependencyNotAvailable()
Expand Down
2 changes: 2 additions & 0 deletions src/diffusers/pipelines/__init__.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -146,6 +146,7 @@
_import_structure["pag"].extend(
[
"AnimateDiffPAGPipeline",
"KolorsPAGPipeline",
"HunyuanDiTPAGPipeline",
"StableDiffusion3PAGPipeline",
"StableDiffusionPAGPipeline",
Expand DownExpand Up@@ -540,6 +541,7 @@
from .pag import (
AnimateDiffPAGPipeline,
HunyuanDiTPAGPipeline,
KolorsPAGPipeline,
PixArtSigmaPAGPipeline,
StableDiffusion3PAGPipeline,
StableDiffusionControlNetPAGPipeline,
Expand Down
2 changes: 2 additions & 0 deletions src/diffusers/pipelines/auto_pipeline.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -162,8 +162,10 @@

if is_sentencepiece_available():
from .kolors import KolorsPipeline
from .pag import KolorsPAGPipeline

AUTO_TEXT2IMAGE_PIPELINES_MAPPING["kolors"] = KolorsPipeline
AUTO_TEXT2IMAGE_PIPELINES_MAPPING["kolors-pag"] = KolorsPAGPipeline
AUTO_IMAGE2IMAGE_PIPELINES_MAPPING["kolors"] = KolorsPipeline

SUPPORTED_TASKS_MAPPINGS = [
Expand Down
12 changes: 12 additions & 0 deletions src/diffusers/pipelines/kolors/tokenizer.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -143,10 +143,18 @@ def get_command(self, token):
def unk_token(self) -> str:
return "<unk>"

@unk_token.setter
def unk_token(self, value: str):
self._unk_token = value
Comment on lines +146 to +148

@sayakpaulsayakpaulAug 7, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like these should have been done in a separate PR?

@asomozaasomozaAug 7, 2024

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if I don't do this, the tests fails (throws AttributeError), I'm ok with doing this in a separate PR but then I'll have to skip the tests in this PR

@sayakpaulsayakpaulAug 7, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah okay. Then no worries.


@property
def pad_token(self) -> str:
return "<unk>"

@pad_token.setter
def pad_token(self, value: str):
self._pad_token = value

@property
def pad_token_id(self):
return self.get_command("<pad>")
Expand All@@ -155,6 +163,10 @@ def pad_token_id(self):
def eos_token(self) -> str:
return "</s>"

@eos_token.setter
def eos_token(self, value: str):
self._eos_token = value

@property
def eos_token_id(self):
return self.get_command("<eos>")
Expand Down
2 changes: 2 additions & 0 deletions src/diffusers/pipelines/pag/__init__.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -25,6 +25,7 @@
_import_structure["pipeline_pag_controlnet_sd"] = ["StableDiffusionControlNetPAGPipeline"]
_import_structure["pipeline_pag_controlnet_sd_xl"] = ["StableDiffusionXLControlNetPAGPipeline"]
_import_structure["pipeline_pag_hunyuandit"] = ["HunyuanDiTPAGPipeline"]
_import_structure["pipeline_pag_kolors"] = ["KolorsPAGPipeline"]
_import_structure["pipeline_pag_pixart_sigma"] = ["PixArtSigmaPAGPipeline"]
_import_structure["pipeline_pag_sd"] = ["StableDiffusionPAGPipeline"]
_import_structure["pipeline_pag_sd_3"] = ["StableDiffusion3PAGPipeline"]
Expand All@@ -44,6 +45,7 @@
from .pipeline_pag_controlnet_sd import StableDiffusionControlNetPAGPipeline
from .pipeline_pag_controlnet_sd_xl import StableDiffusionXLControlNetPAGPipeline
from .pipeline_pag_hunyuandit import HunyuanDiTPAGPipeline
from .pipeline_pag_kolors import KolorsPAGPipeline
from .pipeline_pag_pixart_sigma import PixArtSigmaPAGPipeline
from .pipeline_pag_sd import StableDiffusionPAGPipeline
from .pipeline_pag_sd_3 import StableDiffusion3PAGPipeline
Expand Down
Loading