[FEATURE] LyCORIS support in 2.3

[StAlKeR7779]

Implementation of LyCORIS(extended LoRA), which is 2 formats - LoCon and LoHa(info1, info2).

It's works but i found 2 a bit different implementations of forward function for LoHa. Both works, but I don't know which is better.

2 functions generate same images if remove self.org_module.weight.data addition from LyCORIS implementation, but who's right?

[StAlKeR7779]

@lstein @blessedcoolant

[lstein]

Thanks for the PR!

I just tried this out. It seems to work with some lycoris models and not others:

• A model that works: https://civitai.com/models/20953/soup-nazi-locon-seinfeld
• A model that crashes with a missing hada_t1 key: https://civitai.com/models/27229/smart-falcon-umamusume
• A model that results in latent noise: https://civitai.com/models/14658/kikunojo-one-piece-loha

[lstein]

Thanks for writing this! The code looks good and tests out on all the LyCORIS LoRAs I tried.

[hipsterusername]

@StAlKeR7779 👏