Is there an existing issue for this problem?
Operating system
Windows
GPU vendor
Nvidia (CUDA)
GPU model
No response
GPU VRAM
24GB
Version number
v5.4.2rc1
Browser
chrome
Python dependencies
No response
What happened
Select an SD3.5 main model and a Flux T5 sub-model, the generations work fine. However, when a Flux main model is selected with an SD3.5 T5 sub-model it fails with an assertion error when you try to generate.
What you expected to happen
The T5 models should work both ways. I think the issue is which SD3.5 sub-model is returned to the generation routine. With SD3.5 all sub-models are called the same name so you can't tell from the UI only.
How to reproduce the problem
No response
Additional context
I've not looked into it in depth but it might be related to the textencoder# and tokenizer_# folder definition differences between flux and sd3.5. t5 on flux is #2 but on sd3.5 it is #3. seems like the ui might be displaying the wrong sd3.5 sub model when a flux model is selected which is harder for the user to notice as all the sd3.5 model parts are all called "SD3.5 Size" no matter if it is a vae, t5 or clip model.
If they can all use the same sub models then there is a huge saving in storage space if they are shared. ~10Gb alone for just sd3.5 medium and large is common, so at least that could be saved. Then you also have textencoders and tokenizers for flux that seem to be shareable as well.
Discord username
Skunkworxdark
Is there an existing issue for this problem?
Operating system
Windows
GPU vendor
Nvidia (CUDA)
GPU model
No response
GPU VRAM
24GB
Version number
v5.4.2rc1
Browser
chrome
Python dependencies
No response
What happened
Select an SD3.5 main model and a Flux T5 sub-model, the generations work fine. However, when a Flux main model is selected with an SD3.5 T5 sub-model it fails with an assertion error when you try to generate.
What you expected to happen
The T5 models should work both ways. I think the issue is which SD3.5 sub-model is returned to the generation routine. With SD3.5 all sub-models are called the same name so you can't tell from the UI only.
How to reproduce the problem
No response
Additional context
I've not looked into it in depth but it might be related to the textencoder# and tokenizer_# folder definition differences between flux and sd3.5. t5 on flux is #2 but on sd3.5 it is #3. seems like the ui might be displaying the wrong sd3.5 sub model when a flux model is selected which is harder for the user to notice as all the sd3.5 model parts are all called "SD3.5 Size" no matter if it is a vae, t5 or clip model.
If they can all use the same sub models then there is a huge saving in storage space if they are shared. ~10Gb alone for just sd3.5 medium and large is common, so at least that could be saved. Then you also have textencoders and tokenizers for flux that seem to be shareable as well.
Discord username
Skunkworxdark