Skip to content

Merge path_finder_dev into main - #613

Merged
leofang merged 35 commits into
NVIDIA:mainfrom
rwgk:path_finder_search_priority_v2
May 6, 2025
Merged

Merge path_finder_dev into main#613
leofang merged 35 commits into
NVIDIA:mainfrom
rwgk:path_finder_search_priority_v2

Conversation

@rwgk

@rwgkrwgk commented May 6, 2025

Copy link
Copy Markdown
Contributor

Description

Accumulation of PRs #447, #553, #558, #578, #604

Tested also under PR #610

See _path_finder/README.md for background.

rwgkand others added 30 commits April 9, 2025 11:33
* Unmodified copies of:
* https://github.com/NVIDIA/numba-cuda/blob/bf487d78a40eea87f009d636882a5000a7524c95/numba_cuda/numba/cuda/cuda_paths.py
* https://github.com/numba/numba/blob/f0d24824fcd6a454827e3c108882395d00befc04/numba/misc/findlib.py
* Add Forked from URLs.
* Strip down cuda_paths.py to minimum required for `_get_nvvm_path()`
Tested interactively with:
```
import cuda_paths
nvvm_path = cuda_paths._get_nvvm_path()
print(f"{nvvm_path=}")
```
* ruff auto-fixes (NO manual changes)
* Make `get_nvvm_path()` a pubic API (i.e. remove leading underscore).
* Fetch numba-cuda/numba_cuda/numba/cuda/cuda_paths.py from NVIDIA/numba-cuda#155 AS-IS
* ruff format NO MANUAL CHANGES
* Minimal changes to adapt numba-cuda/numba_cuda/numba/cuda/cuda_paths.py from NVIDIA/numba-cuda#155
* Rename ecosystem/cuda_paths.py -> path_finder.py
* Plug cuda.bindings.path_finder into cuda/bindings/_internal/nvvm_linux.pyx
* Plug cuda.bindings.path_finder into cuda/bindings/_internal/nvjitlink_linux.pyx
* Fix `os.path.exists(None)` issue:
```
______________________ ERROR collecting test_nvjitlink.py ______________________
tests/test_nvjitlink.py:62: in <module>
not check_nvjitlink_usable(), reason="nvJitLink not usable, maybe not installed or too old (<12.3)"
tests/test_nvjitlink.py:58: in check_nvjitlink_usable
return inner_nvjitlink._inspect_function_pointer("__nvJitLinkVersion") != 0
cuda/bindings/_internal/nvjitlink.pyx:257: in cuda.bindings._internal.nvjitlink._inspect_function_pointer
???
cuda/bindings/_internal/nvjitlink.pyx:260: in cuda.bindings._internal.nvjitlink._inspect_function_pointer
???
cuda/bindings/_internal/nvjitlink.pyx:208: in cuda.bindings._internal.nvjitlink._inspect_function_pointers
???
cuda/bindings/_internal/nvjitlink.pyx:102: in cuda.bindings._internal.nvjitlink._check_or_init_nvjitlink
???
cuda/bindings/_internal/nvjitlink.pyx:59: in cuda.bindings._internal.nvjitlink.load_library
???
/opt/hostedtoolcache/Python/3.13.2/x64/lib/python3.13/site-packages/cuda/bindings/path_finder.py:312: in get_cuda_paths
"nvvm": _get_nvvm_path(),
/opt/hostedtoolcache/Python/3.13.2/x64/lib/python3.13/site-packages/cuda/bindings/path_finder.py:285: in _get_nvvm_path
by, path = _get_nvvm_path_decision()
/opt/hostedtoolcache/Python/3.13.2/x64/lib/python3.13/site-packages/cuda/bindings/path_finder.py:96: in _get_nvvm_path_decision
if os.path.exists(nvvm_ctk_dir):
<frozen genericpath>:19: in exists
???
E TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType
```
* Fix another `os.path.exists(None)` issue:
```
______________________ ERROR collecting test_nvjitlink.py ______________________
tests/test_nvjitlink.py:62: in <module>
not check_nvjitlink_usable(), reason="nvJitLink not usable, maybe not installed or too old (<12.3)"
tests/test_nvjitlink.py:58: in check_nvjitlink_usable
return inner_nvjitlink._inspect_function_pointer("__nvJitLinkVersion") != 0
cuda/bindings/_internal/nvjitlink.pyx:257: in cuda.bindings._internal.nvjitlink._inspect_function_pointer
???
cuda/bindings/_internal/nvjitlink.pyx:260: in cuda.bindings._internal.nvjitlink._inspect_function_pointer
???
cuda/bindings/_internal/nvjitlink.pyx:208: in cuda.bindings._internal.nvjitlink._inspect_function_pointers
???
cuda/bindings/_internal/nvjitlink.pyx:102: in cuda.bindings._internal.nvjitlink._check_or_init_nvjitlink
???
cuda/bindings/_internal/nvjitlink.pyx:59: in cuda.bindings._internal.nvjitlink.load_library
???
/opt/hostedtoolcache/Python/3.13.2/x64/lib/python3.13/site-packages/cuda/bindings/path_finder.py:313: in get_cuda_paths
"libdevice": _get_libdevice_paths(),
/opt/hostedtoolcache/Python/3.13.2/x64/lib/python3.13/site-packages/cuda/bindings/path_finder.py:126: in _get_libdevice_paths
by, libdir = _get_libdevice_path_decision()
/opt/hostedtoolcache/Python/3.13.2/x64/lib/python3.13/site-packages/cuda/bindings/path_finder.py:73: in _get_libdevice_path_decision
if os.path.exists(libdevice_ctk_dir):
<frozen genericpath>:19: in exists
???
E TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType
```
* Change "/lib64/" → "/lib/" in nvjitlink_linux.pyx
* nvjitlink_linux.pyx load_library() enhancements, mainly to avoid os.path.join(None, "libnvJitLink.so")
* Add missing f-string f
* Add back get_nvjitlink_dso_version_suffix() call.
* pytest -ra -s -v
* Rewrite nvjitlink_linux.pyx load_library() to produce detailed error messages.
* Attach listdir output to "Unable to load" exception message.
* Guard os.listdir() call with os.path.isdir()
* Fix logic error in nvjitlink_linux.pyx load_library()
* Move path_finder.py to _path_finder_utils/cuda_paths.py, import only public functions from new path_finder.py
* Add find_nvidia_dynamic_library() and use from nvjitlink_linux.pyx, nvvm_linux.pyx
* Fix oversight in _find_using_lib_dir()
* Also look for versioned library in _find_using_nvidia_lib_dirs()
* glob.glob() Python 3.9 compatibility
* Reduce build-and-test.yml to Windows-only, Python 3.12 only.
* Comment out `if: ${{ github.repository_owner == nvidia }}`
* Revert "Comment out `if: ${{ github.repository_owner == nvidia }}`"
This reverts commit b0db24f.
* Add back `linux-64` `host-platform`
* Rewrite load_library() in nvjitlink_windows.pyx to use path_finder.find_nvidia_dynamic_library()
* Revert "Rewrite load_library() in nvjitlink_windows.pyx to use path_finder.find_nvidia_dynamic_library()"
This reverts commit 1bb7151.
* Add _inspect_environment() in find_nvidia_dynamic_library.py, call from nvjitlink_windows.pyx, nvvm_windows.pyx
* Add & use _find_dll_using_nvidia_bin_dirs(), _find_dll_using_cudalib_dir()
* Fix silly oversight: forgot to undo experimental change.
* Also reduce test test-linux matrix.
* Reimplement load_library() functions in nvjitlink_windows.pyx, nvvm_windows.pyx to actively use path_finder.find_nvidia_dynamic_library()
* Factor out load_nvidia_dynamic_library() from _internal/nvjitlink_linux.pyx, nvvm_linux.pyx
* Generalize load_nvidia_dynamic_library.py to also work under Windows.
* Add `void*` return type to load_library() implementations in _internal/nvjitlink_windows.pyx, nvvm_windows.pyx
* Resolve cython error: object handle vs `void*` handle
```
Error compiling Cython file:
------------------------------------------------------------
...
err = (<int (*)(int*) nogil>__cuDriverGetVersion)(&driver_ver)
if err != 0:
raise RuntimeError('something went wrong')
# Load library
handle = load_library(driver_ver)
^
------------------------------------------------------------
cuda\bindings\_internal\nvjitlink.pyx:72:29: Cannot convert 'void *' to Python object
```
* Resolve another cython error: `void*` handle vs `intptr_t` handle
```
Error compiling Cython file:
------------------------------------------------------------
...
handle = load_library(driver_ver)
# Load function
global __nvJitLinkCreate
try:
__nvJitLinkCreate = <void*><intptr_t>win32api.GetProcAddress(handle, 'nvJitLinkCreate')
^
------------------------------------------------------------
cuda\bindings\_internal\nvjitlink.pyx:78:73: Cannot convert 'void *' to Python object
```
* Resolve signed/unsigned runtime error. Use uintptr_t consistently.
https://github.com/NVIDIA/cuda-python/actions/runs/14224673173/job/39861750852?pr=447#logs
```
=================================== ERRORS ====================================
_____________________ ERROR collecting test_nvjitlink.py ______________________
tests\test_nvjitlink.py:62: in <module>
not check_nvjitlink_usable(), reason="nvJitLink not usable, maybe not installed or too old (<12.3)"
tests\test_nvjitlink.py:58: in check_nvjitlink_usable
return inner_nvjitlink._inspect_function_pointer("__nvJitLinkVersion") != 0
cuda\\bindings\\_internal\\nvjitlink.pyx:221: in cuda.bindings._internal.nvjitlink._inspect_function_pointer
???
cuda\\bindings\\_internal\\nvjitlink.pyx:224: in cuda.bindings._internal.nvjitlink._inspect_function_pointer
???
cuda\\bindings\\_internal\\nvjitlink.pyx:172: in cuda.bindings._internal.nvjitlink._inspect_function_pointers
???
cuda\\bindings\\_internal\\nvjitlink.pyx:73: in cuda.bindings._internal.nvjitlink._check_or_init_nvjitlink
???
cuda\\bindings\\_internal\\nvjitlink.pyx:46: in cuda.bindings._internal.nvjitlink.load_library
???
E OverflowError: can't convert negative value to size_t
```
* Change <void*><uintptr_t>win32api.GetProcAddress` back to `intptr_t`. Changing load_nvidia_dynamic_library() to also use to-`intptr_t` conversion, for compatibility with win32api.GetProcAddress. Document that CDLL behaves differently (it uses to-`uintptr_t`).
* Use win32api.LoadLibrary() instead of ctypes.windll.kernel32.LoadLibraryW(), to be more similar to original (and working) cython code.
Hoping to resolve this kind of error:
```
_ ERROR at setup of test_c_or_v_program_fail_bad_option[txt-compile_program] __
request = <SubRequest 'minimal_nvvmir' for <Function test_c_or_v_program_fail_bad_option[txt-compile_program]>>
@pytest.fixture(params=MINIMAL_NVVMIR_FIXTURE_PARAMS)
def minimal_nvvmir(request):
for pass_counter in range(2):
nvvmir = MINIMAL_NVVMIR_CACHE.get(request.param, -1)
if nvvmir != -1:
if nvvmir is None:
pytest.skip(f"UNAVAILABLE: {request.param}")
return nvvmir
if pass_counter:
raise AssertionError("This code path is meant to be unreachable.")
# Build cache entries, then try again (above).
> major, minor, debug_major, debug_minor = nvvm.ir_version()
tests\test_nvvm.py:148:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cuda\bindings\nvvm.pyx:95: in cuda.bindings.nvvm.ir_version
cpdef tuple ir_version():
cuda\bindings\nvvm.pyx:113: in cuda.bindings.nvvm.ir_version
status = nvvmIRVersion(&major_ir, &minor_ir, &major_dbg, &minor_dbg)
cuda\bindings\cynvvm.pyx:19: in cuda.bindings.cynvvm.nvvmIRVersion
return _nvvm._nvvmIRVersion(majorIR, minorIR, majorDbg, minorDbg)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E cuda.bindings._internal.utils.FunctionNotFoundError: function nvvmIRVersion is not found
```
* Remove debug print statements.
* Remove some cruft.
* Trivial renaming of variables. No functional changes.
* Revert debug changes under .github/workflows
* Rename _path_finder_utils → _path_finder
* Remove LD_LIBRARY_PATH in fetch_ctk/action.yml
* Linux: First try using the platform-specific dynamic loader search mechanisms
* Add _windows_load_with_dll_basename()
* Revert "Revert debug changes under .github/workflows"
This reverts commit cc6113c.
* Add debug prints in load_nvidia_dynamic_library()
* Report dlopen error for libnvrtc.so.12
* print("\nLOOOK dlfcn.dlopen('libnvrtc.so.12', dlfcn.RTLD_NOW)", flush=True)
* Revert "Remove LD_LIBRARY_PATH in fetch_ctk/action.yml"
This reverts commit 1b1139c.
* Only remove ${CUDA_PATH}/nvvm/lib64 from LD_LIBRARY_PATH
* Use path_finder.load_nvidia_dynamic_library("nvrtc") from cuda/bindings/_bindings/cynvrtc.pyx.in
* Somewhat ad hoc heuristics for nvidia_cuda_nvrtc wheels.
* Remove LD_LIBRARY_PATH entirely from .github/actions/fetch_ctk/action.yml
* Remove CUDA_PATH\nvvm\bin in .github/workflows/test-wheel-windows.yml
* Revert "Remove LD_LIBRARY_PATH entirely from .github/actions/fetch_ctk/action.yml"
This reverts commit bff8cf0.
* Revert "Somewhat ad hoc heuristics for nvidia_cuda_nvrtc wheels."
This reverts commit 43abec8.
* Restore cuda/bindings/_bindings/cynvrtc.pyx.in as-is on main
* Remove debug print from load_nvidia_dynamic_library.py
* Reapply "Revert debug changes under .github/workflows"
This reverts commit aaa6aff.
* Revert "Restore cuda/bindings/_bindings/cynvrtc.pyx.in as-is on main"
This reverts commit ba093f5.
* Revert "Reapply "Revert debug changes under .github/workflows""
This reverts commit 8f69f83.
* Also load nvrtc from cuda_bindings/tests/path_finder.py
* Add heuristics for nvidia_cuda_nvrtc Windows wheels.
Also fix a couple bugs discovered by ChatGPT:
* `glob.glob()` in this code return absolute paths.
* stray `error_messages = []`
* Add debug prints, mostly for `os.add_dll_directory(bin_dir)`
* Fix unfortunate silly oversight (import os missing under Windows)
* Use `win32api.LoadLibraryEx()` with suitable `flags`; also update `os.environ["PATH"]`
* Hard-wire WinBase.h constants (they are not exposed by win32con)
* Remove debug prints
* Reapply "Reapply "Revert debug changes under .github/workflows""
This reverts commit b002ff6.
* Revert "Reapply "Revert debug changes under .github/workflows""
This reverts commit 8f69f83.
* Add names of all CTK 12.8.1 x86_64-linux libraries (.so) as `path_finder.SUPPORTED_LIBNAMES`
https://chatgpt.com/share/67f98d0b-148c-8008-9951-9995cf5d860c
* Add `SUPPORTED_WINDOWS_DLLS`
* Add copyright notice
* Move SUPPORTED_LIBNAMES, SUPPORTED_WINDOWS_DLLS to _path_finder/supported_libs.py
* Use SUPPORTED_WINDOWS_DLLS in _windows_load_with_dll_basename()
* Change "Set up mini CTK" to use `method: local`, remove `sub-packages` line.
* Use Jimver/cuda-toolkit@v0.2.21 also under Linux, `method: local`, no `sub-packages`.
* Add more `nvidia-*-cu12` wheels to get as many of the supported shared libraries as possible.
* Revert "Use Jimver/cuda-toolkit@v0.2.21 also under Linux, `method: local`, no `sub-packages`."
This reverts commit d499806.
Problem observed:
```
/usr/bin/docker exec 1b42cd4ea3149ac3f2448eae830190ee62289b7304a73f8001e90cead5005102 sh -c "cat /etc/*release | grep ^ID"
Warning: Failed to restore: Cache service responded with 422
/usr/bin/tar --posix -cf cache.tgz --exclude cache.tgz -P -C /__w/cuda-python/cuda-python --files-from manifest.txt -z
Failed to save: Unable to reserve cache with key cuda_installer-linux-5.15.0-135-generic-x64-12.8.0, another job may be creating this cache. More details: This legacy service is shutting down, effective April 15, 2025. Migrate to the new service ASAP. For more information: https://gh.io/gha-cache-sunset
Warning: Error during installation: Error: Unable to locate executable file: sudo. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.
Error: Error: Unable to locate executable file: sudo. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.
```
* Change test_path_finder::test_find_and_load() to skip cufile on Windows, and report exceptions as failures, except for cudart
* Add nvidia-cuda-runtime-cu12 to pyproject.toml (for libname cudart)
* test_path_finder.py: before loading cusolver, load nvJitLink, cusparse, cublas (experiment to see if that resolves the only Windows failure)
Test (win-64, Python 3.12, CUDA 12.8.0, Runner default, CTK wheels) / test
```
================================== FAILURES ===================================
________________________ test_find_and_load[cusolver] _________________________
libname = 'cusolver'
@pytest.mark.parametrize("libname", path_finder.SUPPORTED_LIBNAMES)
def test_find_and_load(libname):
if sys.platform == "win32" and libname == "cufile":
pytest.skip(f'test_find_and_load("{libname}") not supported on this platform')
print(f'\ntest_find_and_load("{libname}")')
failures = []
for algo, func in (
("find", path_finder.find_nvidia_dynamic_library),
("load", path_finder.load_nvidia_dynamic_library),
):
try:
out = func(libname)
except Exception as e:
out = f"EXCEPTION: {type(e)} {str(e)}"
failures.append(algo)
print(out)
print()
> assert not failures
E AssertionError: assert not ['load']
tests\test_path_finder.py:29: AssertionError
```
* test_path_finder.py: load *only* nvJitLink before loading cusolver
* Run each test_find_or_load_nvidia_dynamic_library() subtest in a subprocess
* Add cublasLt to supported_libs.py and load deps for cusolver, cusolverMg, cusparse in test_path_finder.py. Also restrict test_path_finder.py to test load only for now.
* Add supported_libs.DIRECT_DEPENDENCIES
* Remove cufile_rdma from supported libs (comment out).
https://chatgpt.com/share/68033a33-385c-8008-a293-4c8cc3ea23ae
* Split out `PARTIALLY_SUPPORTED_LIBNAMES`. Fix up test code.
* Reduce public API to only load_nvidia_dynamic_library, SUPPORTED_LIBNAMES
* Set CUDA_BINDINGS_PATH_FINDER_TEST_ALL_LIBNAMES=1 to match expected availability of nvidia shared libraries.
* Refactor as `class _find_nvidia_dynamic_library`
* Strict wheel, conda, system rule: try using the platform-specific dynamic loader search mechanisms only last
* Introduce _load_and_report_path_linux(), add supported_libs.EXPECTED_LIB_SYMBOLS
* Plug in ctypes.windll.kernel32.GetModuleFileNameW()
* Keep track of nvrtc-related GitHub comment
* Factor out `_find_dll_under_dir(dirpath, file_wild)` and reuse from `_find_dll_using_nvidia_bin_dirs()`, `_find_dll_using_cudalib_dir()` (to fix loading nvrtc64_120_0.dll from local CTK)
* Minimal "is already loaded" code.
* Add THIS FILE NEEDS TO BE REVIEWED/UPDATED FOR EACH CTK RELEASE comment in _path_finder/supported_libs.py
* Add SUPPORTED_LINUX_SONAMES in _path_finder/supported_libs.py
* Update SUPPORTED_WINDOWS_DLLS in _path_finder/supported_libs.py based on DLLs found in cuda_*win*.exe files.
* Remove `os.add_dll_directory()` and `os.environ["PATH"]` manipulations from find_nvidia_dynamic_library.py. Add `supported_libs.LIBNAMES_REQUIRING_OS_ADD_DLL_DIRECTORY` and use from `load_nvidia_dynamic_library()`.
* Move nvrtc-specific code from find_nvidia_dynamic_library.py to `supported_libs.is_suppressed_dll_file()`
* Introduce dataclass LoadedDL as return type for load_nvidia_dynamic_library()
* Factor out _abs_path_for_dynamic_library_* and use on handle obtained through "is already loaded" checks
* Factor out _load_nvidia_dynamic_library_no_cache() and use for exercising LoadedDL.was_already_loaded_from_elsewhere
* _check_nvjitlink_usable() in test_path_finder.py
* Undo changes in .github/workflows/ and cuda_bindings/pyproject.toml
* Move cuda_bindings/tests/path_finder.py -> toolshed/run_cuda_bindings_path_finder.py
* Add bandit suppressions in test_path_finder.py
* Add pytest info_summary_append fixture and use from test_path_finder.py to report the absolute paths of the loaded libraries.
* Undo changes to the nvJitLink, nvrtc, nvvm bindings
* Undo changes under .github, specific to nvvm, manipulating LD_LIBRARY_PATH or PATH
* PARTIALLY_SUPPORTED_LIBNAMES_LINUX, PARTIALLY_SUPPORTED_LIBNAMES_WINDOWS
* Update EXPECTED_LIB_SYMBOLS for nvJitLink to cleanly support CTK versions 12.0, 12.1, 12.2
* Save result of factoring out load_dl_common.py, load_dl_linux.py, load_dl_windows.py with the help of Cursor.
* Fix an auto-generated docstring
* first round of Cursor refactoring (about 4 iterations until all tests passed), followed by ruff auto-fixes
* Revert "first round of Cursor refactoring (about 4 iterations until all tests passed), followed by ruff auto-fixes"
This reverts commit 001a6a2.
There were many GitHub Actions jobs that failed (all tests with 12.x):
https://github.com/NVIDIA/cuda-python/actions/runs/14677553387
This is not worth spending time debugging.
Especially because
* Cursor has been unresponsive for at least half an hour:
We're having trouble connecting to the model provider. This might be temporary - please try again in a moment.
* The refactored code does not seem easier to read.
* A couple trivial tweaks
* Prefix the public API (just two items) with underscores for now.
* Add SPDX-License-Identifier to all files under toolshed/ that don't have it already
* Add SPDX-License-Identifier under cuda_bindings/tests/
* Respond to "Do these need to be run as subprocesses?" review question (NVIDIA#578 (comment))
* Respond to "dead code?" review questions (e.g. NVIDIA#578 (comment))
* Respond to "Do we need to implement a cache separately ..." review question (NVIDIA#578 (comment))
* Remove cuDriverGetVersion() function for now.
* Move add_dll_directory() from load_dl_common.py to load_dl_windows.py (response to review question NVIDIA#578 (comment))
* Add SPDX-License-Identifier and # Forked from: URL in cuda_paths.py
* Add Add SPDX-License-Identifier and Original LICENSE in findlib.py
* Very first draft of README.md
* Update README.md, mostly as revised by perplexity, with various manual edits.
* Refork cuda_paths.py AS-IS: https://github.com/NVIDIA/numba-cuda/blob/8c9c9d0cb901c06774a9abea6d12b6a4b0287e5e/numba_cuda/numba/cuda/cuda_paths.py
* ruff format cuda_paths.py (NO manual changes)
* Add back _get_numba_CUDA_INCLUDE_PATH from 2279bda (i.e. cuda_paths.py as it was right before re-forking)
* Remove cuda_paths.py dependency on numba.cuda.cudadrv.runtime
* Add Forked from URLs, two SPDX-License-Identifier, Original Numba LICENSE
* Temporarily restore debug changes under .github/workflows, for expanded path_finder test coverage
* Restore cuda_path.py AS-IT-WAS at commit 2279bda
* Revert "Restore cuda_path.py AS-IT-WAS at commit 2279bda"
This reverts commit 1b88ec2.
* Force compute-sanitizer off unconditionally
* Revert "Force compute-sanitizer off unconditionally"
This reverts commit 2bc7ef6.
* Add timeout=10 seconds to test_path_finder.py subprocess.run() invocations.
* Increase test_path_finder.py subprocess.run() timeout to 30 seconds:
Under Windows, loading cublas or cusolver may exceed the 10 second timeout:
NVIDIA#578 (comment)
* Revert "Temporarily restore debug changes under .github/workflows, for expanded path_finder test coverage"
This reverts commit 47ad79f.
* Force compute-sanitizer off unconditionally
* Add: Note that the search is done on a per-library basis.
* Add Note for CUDA_HOME / CUDA_PATH
* Add 0. **Check if a library was loaded into the process already by some other means.**
* _find_dll_using_nvidia_bin_dirs(): reuse lib_searched_for in place of file_wild
* Systematically replace all relative imports with absolute imports.
* handle: int → ctypes.CDLL fix
* Make load_dl_windows.py abs_path_for_dynamic_library() implementation maximally robust.
* Change argument name → libname for self-consistency
* Systematically replace previously overlooked relative imports with absolute imports.
* Simplify code (also for self-consistency)
* Expand the 3. **System Installations** section with information produced by perplexity
* Pull out `**Environment variables**` into an added section, after manual inspection of cuda_paths.py. Minor additional edits.
* Revert "Force compute-sanitizer off unconditionally"
This reverts commit aeaf4f0.
* Move _path_finder/sys_path_find_sub_dirs.py → find_sub_dirs.py, use find_sub_dirs_all_sitepackages() from find_nvidia_dynamic_library.py
* WIP (search priority updated in README.md but not in code)
* Revert "WIP (search priority updated in README.md but not in code)"
This reverts commit bf9734c.
The previous implementation checked result_queue.empty() before calling get(),
which introduces a classic race condition: the queue may become non-empty
immediately after the check, resulting in missed results or misleading errors.
This patch replaces the empty() check with result_queue.get(timeout=1.0),
allowing the parent process to robustly wait for results with a bounded delay.
Also switches from ctx.SimpleQueue() to ctx.Queue() for compatibility with
timeout-based get(), which SimpleQueue does not support on Python ≤3.12.
Note: The race condition was discovered by Gemini 2.5
rwgk added 4 commits May 5, 2025 21:39
…returns an incompatible `handle`. Use win32api.LoadLibraryEx() instead to ensure self-consistency.
…od. Move run_python_code_safely.py to test/ directory.
@copy-pr-bot

Copy link
Copy Markdown
Contributor

Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@rwgkrwgk changed the title Path finder search priority v2Merge path_finder_dev into mainMay 6, 2025
@rwgk

rwgk commented May 6, 2025

Copy link
Copy Markdown
ContributorAuthor

/ok to test 5fb2d1f

@github-actions

This comment has been minimized.

@rwgkrwgk closed this May 6, 2025
@rwgk
rwgk deleted the path_finder_search_priority_v2 branch May 6, 2025 16:53
@rwgk
rwgk restored the path_finder_search_priority_v2 branch May 6, 2025 16:55
@rwgkrwgk reopened this May 6, 2025
@leofangleofang added P0 High priority - Must do! feature New feature or request cuda.bindings Everything related to the cuda.bindings module labels May 6, 2025
@leofangleofang added this to the cuda-python 12.9.0 & 11.8.7 milestone May 6, 2025
@leofang
leofang merged commit 34ef825 into NVIDIA:mainMay 6, 2025
@leofang

Copy link
Copy Markdown
Member

Great milestone, Ralf! Thanks for putting all of these together!

@rwgk
rwgk deleted the path_finder_search_priority_v2 branch May 6, 2025 17:51
@github-actions

Copy link
Copy Markdown
Doc Preview CI
Preview removed because the pull request was closed or merged.

@rwgk

rwgk commented May 6, 2025

Copy link
Copy Markdown
ContributorAuthor

Tracking commands used to clean up the path_finder_dev branch:

git switch main
git pull
git switch path_finder_dev
git merge main
git branch --unset-upstream
git push origin path_finder_dev:archive/path_finder_dev_after_pr613_was_merged
git push upstream --delete path_finder_dev

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cuda.bindingsEverything related to the cuda.bindings modulefeatureNew feature or requestP0High priority - Must do!

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@rwgk@leofang@kkraus14