Windows doesn't support cudaMemPrefetchAsync()

[stoperro]

Also memory oversubscription is not supported https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#system-requirements which I presume means paged optimizer that overcomes memory spikes won't work on windows.

This results in below error in QLoRA training:

rror invalid device ordinal at line 359 in file F:\Buildy\bitsandbytes_acpopescu\csrc\pythonInterface.c
C:\arrow\cpp\src\arrow\filesystem\s3fs.cc:2598:  arrow::fs::FinalizeS3 was not called even though S3 was initialized.  This could lead to a segmentation fault at exit

(note: above wasn't caused by old transformers version)

[stoperro]

Above linked solution that works for me - checking capabilities for that feature before running the call as hinted in https://stackoverflow.com/a/43430831/950131 . The call is fast, so maybe no need to cache the answer.

[jllllll]

Does this issue only apply to QLoRA training?

[phalexo]

Since I am unable to rebuild bitsandbytes because of Maxwell architecture incompatibility with synchronization primitives, I am trying a different solution, i.e. trapping SIGSEGV signal.

I has not dumped core yet, but I am not sure what it is doing. Python seems to be running but I don't see any activity on the GPUs for about one hour either.

I am running on Ubuntu 20.04, not Windows. So it may be a wider issue than the OS.

[TimDettmers]

This is a duplicate of #477, please redirect all discussion there.

TL;DR: I need to think if I will support Maxwell or not. There might be a workaround for Maxwell support by excluding Paged Optimziers.

[TimDettmers]

This has been fixed and pushed to pip. Memory problems might remain, but these are Windows-specific and there is nothing I can do about that. Thank you for the fix @stoperro , this was an important bugfix.