DiffEngine Plan in 2023

[okotaku]

If you have any other suggestions, please leave comments!

Fix library cores

• Support new config style of MMEngine. example. [Feature] Support Pure Python style Configuration File open-mmlab/mmengine#1071 [Feature] Pure python config #121

Support new methods

• ControlNet
• Multiple input size training like SDXL and NovelAI Aspect Ratio Bucketing. [Feature] Support multi aspectratio resize #50
• LoRA for Convolution Layers like LyCORIS. [Feature] Support Peft #92
• ControlNet Small / Mid. https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0-small [Feature] Support ControlNet Small #49
• Distill SD training. [Feature] Support Distill SDXL training #67
• Instruct Pix2Pix. [Feature] Support InstructPix2Pix #89
• T2I Adapter. [Feature] Support T2I Adapter #64
• IP Adapter [Feature] IP Adapter #59
• Training Diffusion Models with Reinforcement Learning https://huggingface.co/blog/trl-ddpo
• SSD-1B: A Leap in Efficient T2I Generation [Feature] Support SSD-1B #83

Support Datasets

• Google DreamBooth. [Feature] Support google/dreambooth datasets #52
• web dataset. https://huggingface.co/docs/hub/datasets-webdataset

Create new LoRA

• Classic Anime Expressions with SDXL. [Project] Face Expression #46 [Enhance] Update face expression project sdxl lora #48

Create new DreamBooth

• Human face with Distill SD.