klms support for img2img

[bmaltais]

I found this working implementation of klms for img2img... but this project has moved too far away from the original implementation for me to even know where to start to weave this in the current code format...

But here is how it can be done and maybe you can weave this in the code for klms support: https://github.com/lstein/stable-diffusion.git

Edit:

Proper link: https://github.com/hlky/stable-diffusion/blob/main/scripts/img2img_k.py

[lstein]

You sent a link to this repository. Please check.

[bmaltais]

Dang. my bad. Here is the right link: https://github.com/hlky/stable-diffusion/blob/main/scripts/img2img_k.py

[tildebyte]

It seems like, after #12 (I guess?) k_lms support is merged and working. python scripts/dream.py --help (from today's main/HEAD) shows

*snip*
  --sampler {plms,ddim,klms}, -m {plms,ddim,klms}
                        which sampler to use (klms) - can only be set on command line
*snip*

and, indeed, doing identical generations using --sampler plms vs. --sampler klms yields images which are visibly (although subtly) different

[lstein]

Actually the ability to select different samplers is only working for text2img. It goes back to the original DDIMS method when using img2img. This is how it was set up in the CompViz img2img script and I've not dug deeply into it, since sampling differences seem to be less of an issue when there is an init_img.

[Oceanswave]

Sorta related - saw this analysis - https://i.redd.it/ud12agb7goj91.jpg and immediately flipped my default to k_euler and yes indeed I don't get the multi-colored artifacts.

To the point of the post, yeah the init_img is less affected by sampler choice, and since ddim is actually pretty good already...

[lstein]

This feature was folded in a couple of days ago. See 41b26e0

[lstein]

Actually the k_lms sampler is only active when you do a text prompt. If you include an image (i.e. img2img) it falls back to ddims. The img2img code is quite different from the txt2img code, and I didn't feel comfortable attempting this on my own. But, yeah, if you do text prompts, you'll get the better k_lms sampler. Lincoln

…

On Tue, Aug 23, 2022 at 3:10 PM tildebyte ***@***.***> wrote: It seems like, after #12 <#12> (I guess?) k_lms support is merged and working. python scripts/dream.py --help (from today's main/HEAD) shows *snip* --sampler {plms,ddim,klms}, -m {plms,ddim,klms} which sampler to use (klms) - can only be set on command line *snip* and, indeed, doing identical generations using --sampler plms vs. --sampler klms yields images which are *visibly* (although subtly) different — Reply to this email directly, view it on GitHub <#36 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA3EVMDBQVZQ4K2RJ3Z4DTV2UO3RANCNFSM57LZY5HA> . You are receiving this because you commented.Message ID: ***@***.***>

-- *Lincoln Stein* Head, Adaptive Oncology, OICR Senior Principal Investigator, OICR Professor, Department of Molecular Genetics, University of Toronto Tel: 416-673-8514 Cell: 416-817-8240 ***@***.*** *E**xecutive Assistant* Michelle Xin Tel: 647-260-7927 ***@***.*** ***@***.***>* *Ontario Institute for Cancer Research* MaRS Centre, 661 University Avenue, Suite 510, Toronto, Ontario, Canada M5G 0A3 @OICR_news <https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Foicr_news&data=04%7C01%7CMichelle.Xin%40oicr.on.ca%7C9fa8636ff38b4a60ff5a08d926dd2113%7C9df949f8a6eb419d9caa1f8c83db674f%7C0%7C0%7C637583553462287559%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PS9KzggzFoecbbt%2BZQyhkWkQo9D0hHiiujsbP7Idv4s%3D&reserved=0> | www.oicr.on.ca *Collaborate. Translate. Change lives.* This message and any attachments may contain confidential and/or privileged information for the sole use of the intended recipient. Any review or distribution by anyone other than the person for whom it was originally intended is strictly prohibited. If you have received this message in error, please contact the sender and delete all copies. Opinions, conclusions or other information contained in this message may not be that of the organization.