Library bindings are not importable if the underlying library is not installed

[leofang]

During the investigation of #920 I noticed that if I remove nvJitLink from my current environment, cuda.core cannot fall back to use the cuLink* driver APIs because the pathfinder error is not caught:

Traceback (most recent call last):
 File "/local/home/leof/dev/debug_920.py", line 94, in <module>
 program = Program(code, code_type="ptx", options=options)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/local/home/leof/dev/cuda-python/cuda_core/cuda/core/experimental/_program.py", line 411, in __init__
 ObjectCode._init(code.encode(), code_type), options=self._translate_program_options(options)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/local/home/leof/dev/cuda-python/cuda_core/cuda/core/experimental/_program.py", line 420, in _translate_program_options
 return LinkerOptions(
 ^^^^^^^^^^^^^^
 File "<string>", line 24, in __init__
 File "/local/home/leof/dev/cuda-python/cuda_core/cuda/core/experimental/_linker.py", line 189, in __post_init__
 _lazy_init()
 File "/local/home/leof/dev/cuda-python/cuda_core/cuda/core/experimental/_linker.py", line 69, in _lazy_init
 _decide_nvjitlink_or_driver()
 File "/local/home/leof/dev/cuda-python/cuda_core/cuda/core/experimental/_linker.py", line 47, in _decide_nvjitlink_or_driver
 if inner_nvjitlink._inspect_function_pointer("__nvJitLinkVersion") == 0:
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "cuda/bindings/_internal/nvjitlink.pyx", line 247, in cuda.bindings._internal.nvjitlink._inspect_function_pointer
 File "cuda/bindings/_internal/nvjitlink.pyx", line 250, in cuda.bindings._internal.nvjitlink._inspect_function_pointer
 File "cuda/bindings/_internal/nvjitlink.pyx", line 198, in cuda.bindings._internal.nvjitlink._inspect_function_pointers
 File "cuda/bindings/_internal/nvjitlink.pyx", line 70, in cuda.bindings._internal.nvjitlink._check_or_init_nvjitlink
 File "cuda/bindings/_internal/nvjitlink.pyx", line 92, in cuda.bindings._internal.nvjitlink._check_or_init_nvjitlink
 File "cuda/bindings/_internal/nvjitlink.pyx", line 58, in cuda.bindings._internal.nvjitlink.load_library
 File "/local/home/leof/miniforge3/envs/py312_cu129/lib/python3.12/site-packages/cuda/pathfinder/_dynamic_libs/load_nvidia_dynamic_lib.py", line 120, in load_nvidia_dynamic_lib
 return _load_lib_no_cache(libname)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/local/home/leof/miniforge3/envs/py312_cu129/lib/python3.12/site-packages/cuda/pathfinder/_dynamic_libs/load_nvidia_dynamic_lib.py", line 48, in _load_lib_no_cache
 found.raise_if_abs_path_is_None()
 File "/local/home/leof/miniforge3/envs/py312_cu129/lib/python3.12/site-packages/cuda/pathfinder/_dynamic_libs/find_nvidia_dynamic_lib.py", line 199, in raise_if_abs_path_is_None
 raise DynamicLibNotFoundError(f'Failure finding "{self.lib_searched_for}": {err}\n{att}')
cuda.pathfinder._dynamic_libs.load_dl_common.DynamicLibNotFoundError: Failure finding "libnvJitLink.so": No such file: libnvJitLink.so*, No such file: libnvJitLink.so*

This echos @kkraus14's concern earlier (#728). We want to be able to catch errors at run time when the underlying library (including the driver) does not provide the needed symbol, which implies that the exception raising should be deferred to as late as when it's actually needed.

[leofang]

I think load_library should be written as

cdef void* load_library(int driver_ver) except* with gil:
 from cuda.pathfinder import DynamicLibNotFoundError
 cdef uintptr_t handle
 try:
 handle = load_nvidia_dynamic_lib("nvJitLink")._handle_uint
 except DynamicLibNotFoundError:
 handle = 0
 return <void*>handle

cc @mdboom @rwgk for vis

[leofang]

@rwgk confirmed that we have a regression in cuda-bindings:

• 12.8.0:

RuntimeError: Failed to load nvJitLink

• 12.9.2:

cuda.pathfinder._dynamic_libs.load_dl_common.DynamicLibNotFoundError: Failure finding "nvJitLink*.dll": No such file: nvJitLink*.dll, No such file: nvJitLink*.dll

[mdboom]

Looking at this more closely, I'm not sure it's a regression, at least at this level.

I compared the behavior of main to the 12.8.0 tag.

In both cases, it was possible to from cuda.bindings import nvjitlink, even if the nvjitlink .so is missing. It's only upon making the first API call that there is an exception. In 12.8.0 this was a RuntimeError and in main it's a cuda.pathfinder._dynamic_libs.load_dl_common.DynamicLibNotFoundError, which is a subclass of RuntimeError, which I don't think is an API change.

The fallback behavior to use the driver is determined by _linker._decide_nvjitlink_or_driver, which should return True rather than raising an exception. In both main and 12.8.0, it looks like it's throwing an exception.

We could just catch this in _decide_nvjitlink_or_driver and add some testing...?

[leofang]

Thanks, Mike. It appears that this code has never been working. This is with either cuda-python 12.6.2.post1 (in which we secretly shipped the nvJitLink bindings) or cuda-bindings 12.8.0

>>> from cuda.bindings._internal import nvjitlink
>>> nvjitlink._inspect_function_pointer("__nvJitLinkVersion")
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "cuda/bindings/_internal/nvjitlink.pyx", line 254, in cuda.bindings._internal.nvjitlink._inspect_function_pointer
 File "cuda/bindings/_internal/nvjitlink.pyx", line 257, in cuda.bindings._internal.nvjitlink._inspect_function_pointer
 File "cuda/bindings/_internal/nvjitlink.pyx", line 205, in cuda.bindings._internal.nvjitlink._inspect_function_pointers
 File "cuda/bindings/_internal/nvjitlink.pyx", line 99, in cuda.bindings._internal.nvjitlink._check_or_init_nvjitlink
 File "cuda/bindings/_internal/nvjitlink.pyx", line 64, in cuda.bindings._internal.nvjitlink.load_library
RuntimeError: Failed to dlopen libnvJitLink (libnvJitLink.so: cannot open shared object file: No such file or directory)

and what's shown in the OP is with cuda-bindings 12.9.x. I am scratching my head trying to understand what's missing... 🤔 Maybe the fallback to driver with this branch

cuda-python/cuda_core/cuda/core/experimental/_linker.py

Lines 47 to 49 in c4bb623

	if inner_nvjitlink._inspect_function_pointer("__nvJitLinkVersion") == 0:
	# binding is available, but nvJitLink is not installed
	_nvjitlink = None

has never worked or tested... I am certain that the other branch works

cuda-python/cuda_core/cuda/core/experimental/_linker.py

Lines 43 to 45 in c4bb623

	except ImportError:
	# binding is not available
	_nvjitlink = None

because CUDA 12.0 was tested in the CI until very recently when we dropped the inline config from the workflow:

cuda-python/.github/workflows/test-wheel-linux.yml

Lines 64 to 72 in 5a13448

	pull-request:
	- { ARCH: ${ARCH}, PY_VER: '3.9', CUDA_VER: '12.0.1', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
	- { ARCH: ${ARCH}, PY_VER: '3.9', CUDA_VER: '12.9.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'latest' }
	- { ARCH: ${ARCH}, PY_VER: '3.10', CUDA_VER: '12.9.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
	- { ARCH: ${ARCH}, PY_VER: '3.11', CUDA_VER: '12.9.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'latest' }
	- { ARCH: ${ARCH}, PY_VER: '3.12', CUDA_VER: '12.0.1', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'earliest' }
	- { ARCH: ${ARCH}, PY_VER: '3.12', CUDA_VER: '12.9.0', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
	- { ARCH: ${ARCH}, PY_VER: '3.13', CUDA_VER: '12.0.1', LOCAL_CTK: '1', GPU: ${gpu}, DRIVER: 'latest' }
	- { ARCH: ${ARCH}, PY_VER: '3.13', CUDA_VER: '12.9.0', LOCAL_CTK: '0', GPU: ${gpu}, DRIVER: 'latest' }

in favor of a json file. @mdboom @rwgk Would you agree with this assessment and that the bug is in cuda.core not cuda.bindings?