Follow-up to #14304.
enable_auto_cpu_offload recommends group offloading for a model that OOMs with everything else already offloaded ("it does not fit on the device by itself"). but currently we cannot use group offloading and auto offloading togehter, should add support once #14304 is in
Follow-up to #14304.
enable_auto_cpu_offloadrecommends group offloading for a model that OOMs with everything else already offloaded ("it does not fit on the device by itself"). but currently we cannot use group offloading and auto offloading togehter, should add support once #14304 is in