Skip to content

fix(mm): identify Qwen2.5/Qwen3 causal LMs as Text LLM, not Qwen3 encoder - #9097

Merged
lstein merged 9 commits into
invoke-ai:mainfrom
Pfannkuchensack:fix/qwen3-encoder-misidentifies-textllm
May 5, 2026
Merged

fix(mm): identify Qwen2.5/Qwen3 causal LMs as Text LLM, not Qwen3 encoder#9097
lstein merged 9 commits into
invoke-ai:mainfrom
Pfannkuchensack:fix/qwen3-encoder-misidentifies-textllm

Conversation

@Pfannkuchensack

Copy link
Copy Markdown
Member

Summary

Qwen3Encoder_Qwen3Encoder_Config matches any directory with config.json at the root and a Qwen* class name, which also describes a complete causal LM like Qwen2.5-1.5B-Instruct. With both configs matching and equal sort keys, the encoder won the tie-break, blocking the model from being used as a prompt expander.

A standalone text_encoder download has no tokenizer files; a complete causal LM does. Use that to reject the encoder match when tokenizer files are present alongside config.json at the root.

Also fixes the secondary failure when manually switching the type from Qwen3Encoder to Text LLM: the existing record's format=qwen3_encoder and variant were carried over and produced no matching discriminator under text_llm. On ValidationError after a type change, retry with the stale fields stripped so the new class can apply its defaults.

Closes #9090

Related Issues / Discussions

QA Instructions

Primary bug — probe misidentification:

  1. Open Model Manager → Add Model → HuggingFace.
  2. Install Qwen/Qwen2.5-1.5B-Instruct.
  3. Verify the model is identified as Text LLM (not Qwen3 Encoder) and is usable as a prompt expander.

Secondary bug — manual type switch:

  1. On main (without this PR), install Qwen2.5-1.5B-Instruct; it lands as Qwen3 Encoder.
  2. With this PR applied, edit the existing record and change the type to Text LLM → save.
  3. Expected: save succeeds and the record is now a TextLLM_Diffusers_Config. Previously this returned "Model Update Failed".

Regression checks:

  • Install a real Qwen3 encoder (e.g. the text_encoder subfolder bundled with a Z-Image / Qwen-Image pipeline) — should still be identified as Qwen3 Encoder.
  • Install a full diffusers pipeline that contains a text_encoder/ subfolder — unchanged behavior.
  • Run pytest tests/app/services/model_records/ tests/backend/model_manager/configs/ → 31 tests pass.

Merge Plan

Standard merge — no schema/migration changes, no sensitive surfaces touched. Backend-only fix.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

…oder

`Qwen3Encoder_Qwen3Encoder_Config` matches any directory with `config.json`
at the root and a Qwen* class name, which also describes a complete causal
LM like `Qwen2.5-1.5B-Instruct`. With both configs matching and equal sort
keys, the encoder won the tie-break, blocking the model from being used as
a prompt expander.

A standalone text_encoder download has no tokenizer files; a complete causal
LM does. Use that to reject the encoder match when tokenizer files are
present alongside `config.json` at the root.

Also fixes the secondary failure when manually switching the type from
`Qwen3Encoder` to `Text LLM`: the existing record's `format=qwen3_encoder`
and `variant` were carried over and produced no matching discriminator under
`text_llm`. On `ValidationError` after a type change, retry with the stale
fields stripped so the new class can apply its defaults.

Closes invoke-ai#9090
@github-actions github-actions Bot added python PRs that change python files backend PRs that change backend files services PRs that change app services python-tests PRs that change python tests labels Apr 30, 2026

@lstein lstein left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as advertised.

@lstein

lstein commented May 5, 2026

Copy link
Copy Markdown
Collaborator

Identification and reidentification are working properly. Just a note on the secondary bug, that when I install Qwen2.5-1.5B-Instruct before applying the PR, apply the PR, and then try to change its type without also changing its format, I get this error:

[2026-05-04 21:44:27,107]::[InvokeAI]::ERROR --> 1 validation error for TextLLM_Diffusers_Config
format
  Input should be <ModelFormat.Diffusers: 'diffusers'> [type=literal_error, input_value='qwen3_encoder', input_type=str]
    For further information visit https://errors.pydantic.dev/2.12/v/literal_error

I have to change the Model Format to diffusers as well as changing the Model Type, after which all is well. Please let me know if this is the expected behavior before I merge this.

@Pfannkuchensack

Copy link
Copy Markdown
Member Author

Yes, the Format of that Model is diffusers.

…sent

The Expand Prompt (sparkle) and Image-to-Prompt buttons used to disappear
entirely when no Text LLM or LLaVA model was installed, leaving users no way
to discover the feature exists.

Both buttons now stay visible at all times. Clicking them with no suitable
model installed opens a popover that:

- explains what kind of model the feature needs
- recommends a default (Qwen2.5-1.5B-Instruct for prompt expansion, LLaVA
  Onevision 0.5B for image-to-prompt)
- offers an "Open Model Manager" button that jumps straight to the Starter
  Models sub-tab (mirrors the useStarterModelsToast flow, but skips the
  Launchpad detour since we already know which models the user needs).

Tooltip labels also reflect the missing-model state.

Adds four starter models so the recommended setups are one click away:
- Qwen2.5-1.5B-Instruct (~3 GB) — recommended Text LLM default
- Qwen2.5-3B-Instruct (~6 GB) — higher quality Text LLM
- SmolLM2-1.7B-Instruct (~3 GB) — Apache-2.0 Text LLM alternative
- LLaVA Onevision Qwen2 7B (~16 GB) — larger LLaVA option alongside
  the existing 0.5B starter
@Pfannkuchensack

Copy link
Copy Markdown
Member Author
image image

@github-actions github-actions Bot added the frontend PRs that change frontend files label May 5, 2026
@lstein
lstein enabled auto-merge (squash) May 5, 2026 02:29
@lstein
lstein merged commit d50c898 into invoke-ai:main May 5, 2026
16 checks passed
@Pfannkuchensack
Pfannkuchensack deleted the fix/qwen3-encoder-misidentifies-textllm branch May 5, 2026 10:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend PRs that change backend files frontend PRs that change frontend files python PRs that change python files python-tests PRs that change python tests services PRs that change app services v6.13.x

Projects

Status: 6.13.x Theme: MODELS

Development

Successfully merging this pull request may close these issues.

[bug]: Qwen2.5-1.5B-Instruct model not recognized as text_llm

2 participants