Initial import of cuda.core.system

[mdboom]

(Marked as draft as a reminder to not merge until after the 0.5.0 release...)

Prerequisites to get this PR to pass:

• Bump version to 13.1.2 #1404
• Bump version to 12.9.6 #1405
• Updates to cuda.bindings that are prereqs to cuda.core.system #1406
• 12.9.x backport: Updates to cuda.bindings that are prereqs to cuda.core.system #1407
• Use setuptools-scm to determine versions #1411
• Backport #1411 #1415

This is the first landing of cuda.core.system, with all of the features in the nvutil prototype (which sort of has an arbitrary collection of the most core things in NVML, but is a reasonable starting point for a first PR).

This requires a generator change (not yet merged) to include AUTO_LOWPP_* classes in the .pxd file so they can be cimport'ed. I know we don't usually do that, but it seems important to be able to use those high-level bindings and not repeat ourselves. ABI stability there should be ok -- I don't anticipate needing to change anything on the .pxd side of those classes.

Following the nvutil design, this initializes NVML immediately upon import of cuda.core.system. That feels convenient and may be the right choice, but it will be hard to walk that back. Questions the NVML docs don't answer for me: are there any use cases where you would want to init/shutdown NVML repeatedly. The cuda.bindings.nvml tests do this, so I know it works. Is there any harm in init'ing and never shutting down -- we could add an atexit handler, but I don't know if it's required.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[mdboom]

This is a bit confusing. This is an existing API that returns the CUDA version. It really should be called get_cuda_version to avoid confusion, but that would be a breaking change. There is a new API to return the driver version called get_gpu_driver_version below, but that naming isn't great.

[rparolin]

Could we provide those functions for backwards compat and provide a better named alternative that we are willing to support longer term?

[leofang]

It is actually confusing to call it get_cuda_version. This used to return the UMD (user-mode driver) version, e.g. 13.0, 13.1, .... Now that we have access to NVML, we gain in addition the capability of querying the KMD (kernel-mode driver) version, e.g. 580.65.06. So I'd suggest something like

defget_driver_version(kernel_mode: bool=False) ->tuple[int, ...]:

for users to query either UMD or KMD version. Alternatively, we can allow users to query both at once.

[leofang]

In any case, get_cuda_version is too ambiguous. Depending on the context a CUDA version could refer to any of the versions of KMD, UMD, CTK, NVCC, NVRTC, CUDART, ... and we should help avoid spreading such confusion.

[mdboom]

Thanks for the context. Makes sense. I think I've addressed this now in the latest commit.

[mdboom]

/ok to test

[mdboom]

/ok to test

[mdboom]

/ok to test

[mdboom]

/ok to test

[mdboom]

/ok to test

[mdboom]

/ok to test

[mdboom]

/ok to test

[Copilot]

Pull request overview

This PR introduces the new cuda.core.system module that provides system-level GPU information via NVML (NVIDIA Management Library). It replaces the previous singleton System class with a more comprehensive module that offers both backward-compatible functions and new NVML-powered device management capabilities.

Key changes:

• Replaces singleton System class with module-level functions (get_num_devices(), get_driver_version(), etc.)
• Adds comprehensive Device class with NVML-backed properties for device information (architecture, memory, PCI info, etc.)
• Implements automatic NVML initialization on module import with version-gated availability
• Provides utility functions for formatting bytes and unpacking bitmasks

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
cuda_core/tests/test_memory.py	Updates API calls from ccx_system.num_devices property to ccx_system.get_num_devices() function
cuda_core/tests/system/test_system_utils.py	Adds comprehensive tests for utility functions (format_bytes, unpack_bitmask)
cuda_core/tests/system/test_system_system.py	Adds tests for system-level functions (driver versions, device count, process name)
cuda_core/tests/system/test_system_device.py	Adds extensive tests for Device class properties (architecture, memory, PCI info, etc.)
cuda_core/tests/system/test_nvml_context.py	Adds tests for NVML initialization state management across processes
cuda_core/tests/system/conftest.py	Defines NVML version requirements and skip marker for unsupported versions
cuda_core/tests/system/init.py	Empty init file for test module
cuda_core/cuda/core/experimental/system/utils.pyx	Implements utility functions for byte formatting and bitmask unpacking
cuda_core/cuda/core/experimental/system/system.pyx	Implements system-level query functions with NVML and fallback support
cuda_core/cuda/core/experimental/system/device.pyx	Implements Device class with comprehensive GPU properties via NVML
cuda_core/cuda/core/experimental/system/_nvml_context.pyx	Implements thread-safe, per-process NVML initialization logic
cuda_core/cuda/core/experimental/system/init.py	Module entry point with version-gated NVML imports and initialization
cuda_core/cuda/core/experimental/_system.py	Removes deprecated singleton System class
cuda_core/cuda/core/experimental/init.py	Updates imports to use new system module instead of System singleton
cuda_bindings/cuda/bindings/_nvml.pyx	Adds enums and fixes BAR1Memory property naming (breaking change)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[copy-pr-bot]

Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[mdboom]

/ok to test

[rparolin]

"Following the nvutil design, this initializes NVML immediately upon import of cuda.core.system. That feels convenient and may be the right choice, but it will be hard to walk that back."

Even if we decide to ship with a backwards compatible version that initializes on import and never shuts down. It would be if we at least had API that we used internally to test the system init/shutdown for multiple test suites so we can gain confidence that is stable. Then we can always provide an API to people as an alternative to the legacy behaviour.

[mdboom]

/ok to test

[mdboom]

/ok to test

[mdboom]

/ok to test

[mdboom]

/ok to test

[rparolin]

lgtm to me. Just make sure the spelling mistakes that copilot flagged are resolved. Thank you!

[mdboom]

/ok to test

[mdboom]

/ok to test

[github-actions]

Doc Preview CI
Preview removed because the pull request was closed or merged.