[bug]: Memory Management - freeing memory when not generating?

[mcbexx]

Is there an existing issue for this problem?

• I have searched the existing issues

Operating system

Windows

GPU vendor

Nvidia (CUDA)

GPU model

RTX 2070S

GPU VRAM

8GB

Version number

4.2

Browser

Firefox 125.0.3

Python dependencies

No response

What happened

Lately (I wanna say since Invoke 4.0?) I have been running into the issue that I can no longer use my photo editing software (Affinity Photo 1.10.6.1665) along InvokeAI without very noticable lag or even freezes in the photo editor.
I understand that 8 GB of VRAM are not exactly top end for this kind of work, but I haven't run into this problem before as far as I can recall (user since Oct '22).

Now I am wondering if there maybe has been a new memory management introduced with v4.x that could have caused this issue.
Naturally, when running generations, I would fully expect to not be able to do photo edits at the same time (and vice versa, Invoke is choking and it/s are slowing to a crawl when I am editing larger images with complex layers/filters, so I quit out of Affinity for that as it utilizes hardware acceleration), but the stutters and freezes happen when InvokeAI is basically sitting idle.

I have 32 GB of RAM (and a 3900x 12/24 core CPU) Affinity barely hits 2 GB usage even with my bigger files.
I do see that Python is using up to 10 GB (which is mostly the checkpoint model, I assume), total system RAM usage is about 21 of 32 GB as I am typing this - so I assume the VRAM is the bottleneck.

tldr;
Is it possible that Invoke is unecessarily hogging VRAM when idling?
Or is it intentional and required to maintain smooth operation?
In that case, could a setting be implemented that frees up VRAM after generation (at the cost of a small wind up when starting up generations again)?

Maybe it's just me though. Has anyone noticed similar slowdowns in other apps when Invoke is idle?

What you expected to happen

I would like Invoke not to bog down GPU intensive apps when it's idling in the background, unless that is how it needs to be done to run smoothly otherwise 😉

How to reproduce the problem

No response

Additional context

No response

Discord username

No response

[psychedelicious]

Have you tried disabling lazy_offload? It's in the table here: https://invoke-ai.github.io/InvokeAI/features/CONFIGURATION/#invokeai.app.services.config.config_default.InvokeAIAppConfig

[mcbexx]

Ah, thank you, this seems to have fixed it.

It wasn't explicitly enabled in my current config file (it just contained the custom paths to my output dir and the db path), but I can see I have a .bak file from April 2nd that had it enabled. Never had this particular problem before, but I assume that with the introduction of 4.x it has been set to true by default and the memory handling itself changed internally?

[psychedelicious]

It had always been enabled by default, but we were handling the config file a bit awkwardly until v4 - we wrote defaults to the config file. This caused a number of headaches.

When we did the v4 upgrade and updated the config file setup, we only copied config file settings that were not set to the default value. In this case, we skipped lazy_offload for your config because it was set to the default value, so it disappeared from your config file.

As for why you only recently started running into this problem, I'm not sure... The model loading code was totally rewritten for v4, it's possible the lazy offload wasn't actually working correctly before.

Anyways glad that's working now.