feat(model-manager): identify UNet-only SDXL LoRAs (e.g. slider LoRAs)

[Pfannkuchensack]

Summary

Self-attention-only SDXL LoRAs (e.g. Civitai "slider" LoRAs) patch only the UNet and contain no cross-attention (attn2) or text-encoder (lora_te*) keys. lora_token_vector_length() reads the base's context dimension from exactly those keys, so it returns None for such LoRAs and identification fails with "unrecognized token vector length None".

Add a structural fallback: SDXL's UNet has a deep transformer stack (up to 10 transformer blocks) in its lower-resolution attention blocks, so transformer_blocks indices reach >= 2, whereas SD1.x/SD2.x only ever have a single transformer block (index 0) per attention. _state_dict_looks_like_sdxl_unet_lora() detects SDXL from that structure alone.

The regex is anchored on the lora_unet_ key prefix, because that is exactly the key set convert_sdxl_keys_to_diffusers_format() can consume at load time. Keys outside it (diffusers/PEFT unet.….lora_A.weight) also return None from lora_token_vector_length() and so reach this same fallback — identifying those as SDXL would replace a clean install-time rejection with a ValueError: Unrecognized SDXL LoRA key prefix mid-generation. On top of the prefix, a UNet block name is required in either of the two naming conventions kohya sd-scripts emits — Stability-AI (input_blocks_8_1, middle_block_1, output_blocks_0_1) and diffusers (down_blocks_2_attentions_1, mid_block_attentions_0) — which is also what keeps DiT LoRAs (FLUX/Qwen/Z-Image) that use transformer_blocks without UNet blocks from matching.

Known limitation, documented in the docstring: the >= 2 threshold is what separates SDXL from SD1.x/SD2.x, so an SDXL LoRA confined to the 2-transformer-block attentions (down_blocks_1 / up_blocks_1) is not detected — such a file is indistinguishable from SD1/SD2 by block structure alone.

The fallback only runs after the normal token-vector detection returns None, so existing LoRAs are unaffected. Wired into both the LyCORIS and Diffusers base configs.

Related Issues / Discussions

Closes #7709

Note on provenance: #7709 reports Dramatic Lighting Slider (Illustrious). The fixture and QA below use the sibling upload Dramatic Lighting Slider (Pony Illustrious), whose key set was read off the actual file (840 keys, all lora_unet_-prefixed with diffusers block names, all attn1, no attn2, no lora_te*). Both bases are SDXL finetunes and the sliders are from the same series, so the block layout is expected to be identical — worth confirming against 1128288 itself when closing the issue.

QA Instructions

1. Grab a UNet-only SDXL "slider" LoRA whose keys are all lora_unet_..._attn1_... (no attn2/lora_te*), e.g. Dramatic Lighting Slider.
2. On main, installing/scanning it fails to identify with "unrecognized token vector length None".
3. On this branch, install it via the model manager and confirm it is identified as LoRA / StableDiffusionXL / LyCORIS.
4. Use the LoRA in an SDXL generation and confirm it applies correctly.
5. Regression: install a normal SDXL LoRA (with cross-attention + text-encoder keys) and an SD1.5 LoRA and confirm their bases are still detected correctly (unchanged path).
6. Regression: install a diffusers/PEFT-format SDXL LoRA (unet.….lora_A.weight keys, e.g. an HF-hub pytorch_lora_weights.safetensors) and confirm it is still not identified as SDXL — it must stay unidentified rather than install and then fail at generation time.

Automated coverage: tests/backend/model_manager/configs/test_sdxl_slider_lora_identification.py — unit tests for the structural heuristic (positive: the reported LoRA's real 840-key block layout, plus sliders in both kohya naming conventions; negative: SD1/SD2 shallow stacks in both namings, diffusers/PEFT unet.… keys, OMI keys, Qwen, FLUX, the documented shallow-block miss, empty/non-string keys), a check that every key set the heuristic claims is accepted by convert_sdxl_keys_to_diffusers_format(), and end-to-end identification through both the LyCORIS and Diffusers-folder configs (SDXL matches; SD1/SD2 reject with the base-mismatch reason).

uv run --extra cuda --extra test python -m pytest tests/backend/model_manager/configs/ tests/backend/patches/lora_conversions/ --no-cov -q

Merge Plan

Standard merge — no DB schema or redux changes.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• ❗Changes to a redux slice have a corresponding migration
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[lstein]

Thanks for tackling this — the diagnosis (UNet-only LoRAs have no attn2/lora_te* keys, so lora_token_vector_length() returns None) is exactly right, and a structural fallback is the right shape of fix. But I think the heuristic as written both misses the format the reported LoRA is most likely in and newly accepts a format the SDXL loader cannot load. I verified everything below by running it against this branch.

---

Blocker 1 — the heuristic misses the dominant kohya SDXL key naming, so it probably doesn't fix #7709

_SDXL_UNET_ATTENTION_RE anchors on down_blocks|up_blocks|mid_block + attentions. That's the diffusers UNet naming. kohya sd-scripts SDXL training emits Stability-AI naming — lora_unet_input_blocks_7_1_transformer_blocks_9_attn1_to_q, lora_unet_middle_block_1_..., lora_unet_output_blocks_0_1_... — which has no attentions token, and uses middle_block, not mid_block. We consume that format everywhere already: see SDXL_UNET_STABILITY_TO_DIFFUSERS_MAP / convert_sdxl_keys_to_diffusers_format in invokeai/backend/patches/lora_conversions/sdxl_lora_conversion_utils.py.

And the follow-up on #7709 says: "Model maker responded to my question and confirmed the lora was trained on Kohya."

End-to-end through ModelConfigFactory.from_model_on_disk on this branch:

state dict	result
lora_unet_input_blocks_7_1_transformer_blocks_{0..9}_attn1_* (kohya/Stability, attn1-only)	Unknown_Config — still unidentified
lora_unet_down_blocks_2_attentions_1_transformer_blocks_{0..9}_attn1_* (what the new test builds)	LoRA_LyCORIS_SDXL_Config ✅

So only the diffusers-named variant is covered. Because the test fixture is a reconstruction rather than key names read off the real file, this gap isn't visible in CI. Whichever naming the Dramatic Lighting Slider actually uses, the Stability-named half of the problem is untouched — it'd be worth dumping safetensors key names from the actual file to confirm before re-testing.

Blocker 2 — new false positive: diffusers/PEFT SDXL LoRAs now identify, then hard-crash at load

lora_token_vector_length() returns None for every diffusers/PEFT-named SDXL LoRA (unet.….lora_A.weight, text_encoder.… — neither lora_unet_ nor lora_te* prefixed). Those now fall into the new fallback, and unet.down_blocks.2.attentions.1.transformer_blocks.9.… matches the regex.

Confirmed on this branch:

• A single file with unet.down_blocks.2.attentions.1.transformer_blocks.{0,9}.attn1.to_{q,k,v}.lora_{A,B}.weight → identifies as LoRA_LyCORIS_SDXL_Config (it clears _validate_looks_like_lora on the lora_A.weight suffix).

• The same keys in a folder as pytorch_lora_weights.safetensors → identifies as LoRA_Diffusers_SDXL_Config.

• LoRALoader then routes SDXL through convert_sdxl_keys_to_diffusers_format, which raises:

  ValueError: Unrecognized SDXL LoRA key prefix:
  'unet.down_blocks.2.attentions.1.transformer_blocks.0.attn1.to_q.lora_A.weight'.
  

On main these install as Unknown and are inert. On this branch they install as a normal SDXL LoRA and then blow up mid-generation with an opaque ValueError. This is the standard HF-hub pytorch_lora_weights.safetensors shape, so it isn't a contrived input — and test_diffusers_dot_format explicitly blesses that key shape.

Both blockers have one fix

Anchor on ^lora_unet_ — which is precisely the key set the SDXL loader can convert — and add the Stability block names:

_SDXL_UNET_ATTENTION_RE = re.compile(
    r"^lora_unet_"
    r"(?:(?:down_blocks|up_blocks)_\d+_attentions_\d+"
    r"|mid_block_attentions_\d+"
    r"|(?:input_blocks|output_blocks)_\d+_\d+"
    r"|middle_block_\d+)"
    r"_transformer_blocks_(\d+)_"
)

Verified against the same fixtures: kohya-Stability slider ✅, kohya-diffusers slider ✅, SD1.5 (both namings) ✗, diffusers/PEFT ✗, Qwen ✗, FLUX ✗. As a bonus this makes the DiT-false-positive argument in the comment much stronger — lora_unet_ plus a UNet block name is far more specific than the current unanchored .*.

---

Non-blocking

• test_sd1_sd2_configs_reject uses a bare pytest.raises(NotAMatchError), which passes even if SD1/SD2 reject for an unrelated reason (e.g. the LoRA-heuristic gate rather than the base check). Matching on the message would make it assert what it intends.
• The LoRA_Diffusers_Config_Base path is modified but nothing in the new test module exercises LoRA_Diffusers_* — and that's the path where blocker 2 triggers without needing any lora_* key suffix at all.
• The >= 2 threshold means SDXL LoRAs confined to the 2-layer blocks (down_blocks.1 / up_blocks.1, indices 0–1) stay undetected. Fine as a limitation, worth a line in the docstring.

Things I tried to break and couldn't

For the record, so the surviving risk is clear:

• Probe-order shadowing — LoRA_LyCORIS_SDXL_Config is probed before Z-Image/Krea2/Qwen/Wan/Anima. I checked each; none produce down_blocks|up_blocks|mid_block + attentions + transformer_blocks (they're DiTs using blocks.N / bare transformer_blocks.N). No hijack.
• OMI SDXL — OMI source keys are unet.input_blocks.…; the regex doesn't match and LoRA_OMI_SDXL_Config still wins.
• SD1/SD2 regression — both namings max out at transformer_blocks_0, so the fallback never fires. The has_cosmos_dit_* Anima guard still runs ahead of it in the LyCORIS path.
• ControlNet-LLLite / Anima ControlNet (lllite_unet_input_blocks_…) — no match, and ControlNet configs are probed first regardless.
• ReDoS on the .* — single .* followed by literals, linear backtracking, not exploitable.
• Non-string (GGUF int) keys — the isinstance guard holds; verified no crash.
• Existing suites — tests/backend/model_manager/configs/ + tests/backend/patches/: 487 passed, 3 skipped.

[lstein]

Both blockers from the last round are fixed. I re-verified with my own key fixtures rather than the ones in the PR, and everything holds up. LGTM.

Anchoring on ^lora_unet_ is the right fix for the right reason — convert_sdxl_keys_to_diffusers_format() raises on any prefix other than lora_unet_/lora_te1_/lora_te2_, so the heuristic now claims exactly the key set the loader can consume, and adding the Stability-AI block names covers the other kohya convention.

Independent probes with real-world key strings:

kohya SDXL, Stability names (issue #7709 shape)   identified=True    <- blocker 1
diffusers/PEFT unet.… keys                        identified=False   <- blocker 2
kohya SD1.5 / SD2.1 UNet-only                     identified=False
FLUX double_blocks / single_blocks                identified=False
Qwen transformer_blocks_39, SD3 joint_blocks      identified=False
OMI unet.input_blocks.… / bare diffusers          identified=False
threshold  tb=0,1 -> False;  tb=2,9,10 -> True
SDXL refiner depth (tb=3)                         identified=True

29 tests in the file, 495 across configs/ + patches/, 17/17 CI checks green.

Two structural properties make this comfortable to merge: the change is +352 lines with zero deletions, and the fallback sits only in the else / case _ arm after lora_token_vector_length() — so it can only turn a previously-rejected Unknown into SDXL, never alter an identification that works today. The fixture is anatomically correct too: the block table matches SDXL's real UNet (down_1 ×2, down_2 ×10, mid ×10, up_0 ×10, up_1 ×2), and 70 transformer blocks × 4 projections × 3 suffixes is exactly the 840 keys claimed.

Nice touches beyond what I asked for: documenting the >= 2 threshold's known miss for LoRAs confined to the 2-block attentions, asserting the rejection reason in the SD1/SD2 tests rather than just the exception type, and the test_matched_keys_are_convertible_by_the_loader invariant — that last one is the test that would have caught the original false positive.

I've updated the PR description directly, since it had drifted from the implementation and is what lands in the changelog: the summary still described the pre-fix regex ("anchored on the UNet down/up/mid-block + attentions grouping"), the automated-coverage paragraph predated the new tests, and I added a QA step for the diffusers/PEFT regression plus a note on provenance — #7709 reports model 1128288 while the fixture comes from the sibling upload 1105685. Both are SDXL finetunes from the same slider series so the layout should be identical, but worth a quick confirmation against 1128288 itself before the issue is closed.

Footnote, not a finding

The tests assert an "identified ⇒ convertible" invariant. It holds for every realistic input; I only broke it artificially, with a state dict mixing Stability-named and diffusers-named lora_unet_ keys, which identifies and then trips the converter's separate partial-conversion ValueError. No trainer emits both conventions in one file, and main has the same exposure via the token_vector_length == 2048 path, so there's nothing to do here.

Separately, genuinely PEFT-named (unet.…) UNet-only sliders still can't be identified. That is correct today — the converter can't consume them — but it's the natural follow-up if users start reporting diffusers-packaged sliders.