Skip to content

Fix/model cache Qwen/CogView4 cancel repair - #8959

Merged
lstein merged 6 commits into
invoke-ai:mainfrom
JPPhoto:fix/model-cache-qwen-cancel-repair
Mar 15, 2026
Merged

Fix/model cache Qwen/CogView4 cancel repair#8959
lstein merged 6 commits into
invoke-ai:mainfrom
JPPhoto:fix/model-cache-qwen-cancel-repair

Conversation

@JPPhoto

@JPPhoto JPPhoto commented Mar 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes a canceled-run regression where the next Qwen-based or CogView4-based text encoder invocation could fail with a CPU/CUDA device mismatch.

The change adds a repair hook to partially loaded cached models so required non-autocast tensors are restored to the compute device before inference, while keeping model-cache VRAM accounting consistent. The invocation paths now use that repair flow, and a focused cached-model regression test was added.

Related Issues / Discussions

https://discord.com/channels/1020123559063990373/1049495067846524939/1481290964587778273

QA Instructions

  1. Start a generation that uses the Qwen-based text encoder path on a low-VRAM system.
  2. Cancel it while the text encoder is running.
  3. Start another generation with the same model.
  4. Verify the next run no longer fails with Expected all tensors to be on the same device.

Merge Plan

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

@github-actions github-actions Bot added python PRs that change python files invocations PRs that change invocations backend PRs that change backend files python-tests PRs that change python tests labels Mar 11, 2026
@lstein lstein added the v6.12.0 Intended for 6.12.0 release label Mar 11, 2026
@JPPhoto JPPhoto changed the title Fix/model cache qwen cancel repair Fix/model cache Qwen/CogView4 cancel repair Mar 11, 2026

@lstein lstein left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works. I don't see any obvious code problems.

@Pfannkuchensack

Copy link
Copy Markdown
Member

Code review

Found 1 issue:

  1. Invocation node version strings are not bumped in any of the three modified nodes. PR perf(flux2): optimize cache locking in Klein encoder to fix #7513 #8863 (which modified the same flux2_klein_text_encoder.py node) required a version bump before merging — reviewer @Pfannkuchensack requested it and @lstein blocked merge until it was done. PR Fix/model cache Qwen/CogView4 cancel repair #8959 makes behavioral changes (adds repair hooks with runtime side-effects and warning logs) to all three nodes but leaves their version strings unchanged:

category="conditioning",
version="1.0.0",
classification=Classification.Prototype,

category="conditioning",
version="1.1.1",
classification=Classification.Prototype,

category="conditioning",
version="1.1.0",
classification=Classification.Prototype,

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@JPPhoto

JPPhoto commented Mar 11, 2026

Copy link
Copy Markdown
Collaborator Author

Code review

Found 1 issue:

  1. Invocation node version strings are not bumped in any of the three modified nodes.

Internal changes to the node that don't alter its functionality do not require a node version bump, in general. If you are fundamentally changing how a node works, a version bump is a good idea to alert users. If you're altering its external interface (inputs/outputs), it's essential.

@Pfannkuchensack Pfannkuchensack left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works good.

@lstein
lstein merged commit dc5007f into invoke-ai:main Mar 15, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend PRs that change backend files invocations PRs that change invocations python PRs that change python files python-tests PRs that change python tests v6.12.0 Intended for 6.12.0 release

Projects

Status: 6.12.x

Development

Successfully merging this pull request may close these issues.

3 participants