Skip to content

perf(flux2): optimize cache locking in Klein encoder to fix #7513 - #8863

Merged
lstein merged 7 commits into
invoke-ai:mainfrom
girlyoulookthebest:perf/flux2-klein-cache-optimize
Mar 7, 2026
Merged

perf(flux2): optimize cache locking in Klein encoder to fix #7513#8863
lstein merged 7 commits into
invoke-ai:mainfrom
girlyoulookthebest:perf/flux2-klein-cache-optimize

Conversation

@girlyoulookthebest

@girlyoulookthebest girlyoulookthebest commented Feb 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Addresses a race condition in the ModelCache where models are evicted prematurely on low-VRAM systems (Issue #7513).

  • Refactored invoke() to hold a single ExitStack for the entire duration of the node's execution. This ensures the Text Encoder remains locked in RAM while the conditioning data is being saved.

  • Reordered the loading sequence in _encode_prompt. The Text Encoder is now locked immediately upon retrieval, closing the window where it could be evicted while the Tokenizer is being fetched.

Related Issues / Discussions

Issue #7513

QA Instructions

  1. Use a system with tight VRAM where cache eviction is likely.
  2. Run a generation using the Flux2 Klein text encoder.
  3. In the developer log, confirm that the warning no longer appears.
  4. Performance-wise, confirm that the text encoder is loaded once and stays in RAM, eliminating disk reloading churn.

Merge Plan

Safe to merge immediately. This is an architectural optimization restricted to the flux2_klein_text_encoder node and does not affect other pipelines.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

@github-actions github-actions Bot added python PRs that change python files invocations PRs that change invocations labels Feb 8, 2026
@lstein lstein added the v6.12.0 Intended for 6.12.0 release label Feb 20, 2026
Comment thread invokeai/app/invocations/flux2_klein_text_encoder.py

@lstein lstein left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been using this PR for a few days now without encountering any issues. As soon as @Pfannkuchensack change request is implemented I think it is good to go.

@lstein

lstein commented Feb 28, 2026

Copy link
Copy Markdown
Collaborator

@girlyoulookthebest Please update the version of the fixed node as requested by @Pfannkuchensack . Then I can go ahead and merge your fix into the main branch.

@girlyoulookthebest

Copy link
Copy Markdown
Contributor Author

@girlyoulookthebest Please update the version of the fixed node as requested by @Pfannkuchensack . Then I can go ahead and merge your fix into the main branch.

Version has been updated as requested.

@Pfannkuchensack Pfannkuchensack left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job.

@lstein lstein left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@lstein
lstein enabled auto-merge (squash) March 7, 2026 01:35
@lstein
lstein merged commit 3d81eda into invoke-ai:main Mar 7, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

invocations PRs that change invocations python PRs that change python files v6.12.0 Intended for 6.12.0 release

Projects

Status: 6.12.x

Development

Successfully merging this pull request may close these issues.

3 participants