Uh oh!
There was an error while loading. Please reload this page.
feat(sandbox): add GPU sandbox support for WSL2 - #608
Closed
elezar wants to merge 4 commits into
Closed
Conversation
elezarforce-pushed
the
feat/wsl-cdi-spec-watcher
branch
from
March 25, 2026 14:16
2f1232c to
31ea520Compareelezar
marked this pull request as ready for review
March 25, 2026 18:05
elezar
commented
Mar 25, 2026
MemberAuthor
I'm marking this as ready, but it depends on the mentioned PR. |
5 tasks
elezarforce-pushed
the
feat/wsl-cdi-spec-watcher
branch
2 times, most recently
from
April 15, 2026 09:35
ac398f0 to
a7828f8Compareelezarforce-pushed
the
feat/wsl-cdi-spec-watcher
branch
from
April 15, 2026 09:36
a7828f8 to
8865107Compareelezar
commented
Apr 15, 2026
MemberAuthor
/ok-to-test 8865107 |
elezarforce-pushed
the
feat/wsl-cdi-spec-watcher
branch
from
April 16, 2026 07:53
33834f3 to
482aae3Compareelezar
commented
Apr 16, 2026
MemberAuthor
/ok-to-test 482aae3 |
elezarforce-pushed
the
feat/wsl-cdi-spec-watcher
branch
from
April 16, 2026 08:44
64c9d25 to
126b554Compareelezar
commented
Apr 16, 2026
MemberAuthor
/ok-to-test 126b554 |
elezarforce-pushed
the
feat/wsl-cdi-spec-watcher
branch
2 times, most recently
from
April 16, 2026 09:14
9b5317e to
5c01d5bCompareelezar
commented
Apr 16, 2026
MemberAuthor
/ok-to-test 5c01d5b |
elezar
commented
Apr 16, 2026
MemberAuthor
/ok-to-test 2105c21 |
elezarforce-pushed
the
feat/wsl-cdi-spec-watcher
branch
from
April 16, 2026 12:55
2105c21 to
2bff2d8Compareelezar
commented
Apr 16, 2026
MemberAuthor
/ok-to-test 2bff2d8 |
elezarforce-pushed
the
feat/wsl-cdi-spec-watcher
branch
from
April 16, 2026 14:37
2bff2d8 to
482aae3Compareelezar
commented
Apr 16, 2026
| "/dev/nvidia-uvm", | ||
| "/dev/nvidia-uvm-tools", | ||
| "/dev/nvidia-modeset", | ||
| "/dev/dxg", // WSL2: DXG device (GPU via DirectX kernel driver, injected by CDI) |
MemberAuthor
There was a problem hiding this comment.
@pimlock when considering Tegra-based systems as in #625, the list of device nodes (and other paths) is much longer and are also system dependent. As such, I don't think that hardcoding this list would be feasible. Would it be possible to process the container config instead to get a list of device nodes that we would expect to access?
elezarforce-pushed
the
feat/wsl-cdi-spec-watcher
branch
from
April 17, 2026 07:31
482aae3 to
921e95cCompareelezarforce-pushed
the
feat/wsl-cdi-spec-watcher
branch
from
April 24, 2026 12:36
921e95c to
e4e76b5Compareelezar
commented
Apr 24, 2026
MemberAuthor
@pimlock I have updated this PR to use the v0.19.1 release of the Device Plugin instead of a SHA. The e2e test failures seem unrelated to the WSL2 changes (although they may be due to the device plugin version bump). |
v0.19.1 includes WSL2 CDI spec compatibility fixes. See NVIDIA/k8s-device-plugin#1671. Signed-off-by: Evan Lezar <elezar@nvidia.com>
On WSL2, NVIDIA GPUs are exposed through the DXG kernel driver (/dev/dxg) rather than the native nvidia* devices. CDI injects /dev/dxg as the sole GPU device node, plus GPU libraries under /usr/lib/wsl/. has_gpu_devices() previously only checked for /dev/nvidiactl, which does not exist on WSL2, so GPU enrichment never ran. This meant /dev/dxg was never permitted by Landlock and /proc write access (required by CUDA for thread naming) was never granted. Fix by: - Extending has_gpu_devices() to also detect /dev/dxg - Adding /dev/dxg to GPU_BASELINE_READ_WRITE (device nodes need O_RDWR) - Adding /usr/lib/wsl to GPU_BASELINE_READ_ONLY for CDI-injected GPU library bind-mounts that may not be covered by the /usr parent rule across filesystem boundaries The existing path existence check in enrich_proto_baseline_paths() ensures all new entries are silently skipped on native Linux where these paths do not exist. Signed-off-by: Evan Lezar <elezar@nvidia.com>
… checks Add ClusterRole and ClusterRoleBinding so the openshell service account can list nodes at the cluster scope, which is required by the GPU node capacity check in the Kubernetes driver. Signed-off-by: Evan Lezar <elezar@nvidia.com>
elezarforce-pushed
the
feat/wsl-cdi-spec-watcher
branch
from
April 24, 2026 13:35
fb74ed2 to
35935e6CompareSigned-off-by: Evan Lezar <elezar@nvidia.com>
3 tasks
17 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds GPU sandbox support for WSL2-based systems. On WSL2, NVIDIA GPUs are exposed through the DXG kernel driver (
/dev/dxg) rather than the nativenvidia*devices, and GPU libraries are injected by CDI into/usr/lib/wsl/rather than standard Linux paths.Two changes are required:
Device plugin version bump — bumps
ghcr.io/nvidia/k8s-device-plugintov0.19.1, which includes upstream fixes for WSL2 CDI spec compatibility. See wsl: report a single "all" device to kubelet k8s-device-plugin#1671.Landlock baseline —
has_gpu_devices()previously only checked for/dev/nvidiactl, which does not exist on WSL2, so GPU enrichment never ran. This left/dev/dxg(the WSL2 GPU device node) and/procwrite access (required by CUDA for thread naming) unpermitted by Landlock. Fixes by extending GPU detection to also check/dev/dxg, adding it to the read-write baseline, and adding/usr/lib/wslto the read-only baseline for CDI-injected GPU libraries.The existing path existence checks in the enrichment logic ensure all new baseline entries are silently skipped on native Linux where these paths do not exist.
Related Issue
Closes#404
Depends on #495 and #503.
Changes
deploy/kube/gpu-manifests/nvidia-device-plugin-helmchart.yaml: bump device plugin Helm chart tov0.19.1crates/openshell-sandbox/src/lib.rs: extendhas_gpu_devices()to detect/dev/dxg; add/dev/dxgtoGPU_BASELINE_READ_WRITEand/usr/lib/wsltoGPU_BASELINE_READ_ONLYTesting
mise run pre-commitpassestest_gpu_sandbox_reports_available_gpupasses)Checklist