Uh oh!
There was an error while loading. Please reload this page.
driver: mount wsl lib folder for docker-container driver - #3320
Merged
Conversation
Following up on #3295 (comment). |
MemberAuthor
Thanks for your feedback, can you show the output of |
pechersky
commented
Jul 21, 2025
|
MemberAuthor
@pechersky Seems you're not using a development version of buildx to test changes of this PR. You can build it on your side with https://github.com/docker/buildx?tab=readme-ov-file#building: After testing don't forget to remove the binary: |
pechersky
commented
Jul 21, 2025
Great! After deleting the previous builder, and rerunning, I get the expected output, I think. |
crazy-max
commented
Jul 21, 2025
MemberAuthor
Yes looks good great! |
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
pechersky
commented
Jul 21, 2025
Strangely, if I rerun the |
crazy-max
commented
Jul 21, 2025
MemberAuthor
Can you show the output of |
pechersky
commented
Jul 21, 2025
|
pechersky
commented
Jul 21, 2025
Just to note -- if I delete the builder, then recreate it, then rerun the |
crazy-max
commented
Jul 22, 2025
MemberAuthor
Hum I'm not able to repro. First build: Second build: Third build: I'm not sure what would make |
crazy-max
commented
Jul 22, 2025
MemberAuthor
@tonistiigi Btw I think there is an issue with ondemand and autoallow behavior. When I create a builder with nvidia build tag: And inspect it before doing a build I have: After first build then I get: If I restart the Docker Engine and inspecting this builder again I have: I think as the generated CDI specification is loaded from the build, it takes precedence over the OnDemand and therefore the device is not automatically allowed anymore. |
Sorry for the noise. I understand now that the |
crazy-max
commented
Jul 30, 2025
MemberAuthor
It's available within the builder container on device request right before the build.
Great 👍 |
tonistiigi
left a comment
Member
There was a problem hiding this comment.
It seems requesting GPU on container builder creation introduced in #3063 is not enough as some libraries are missing on WSL.
If that is the case, isn't this a bug in Moby if --gpus does not work correctly?
crazy-max
commented
Aug 11, 2025
MemberAuthor
It works but not for nested containers, I will make a repro with dind to make sure of it. |
tonistiigi
commented
Aug 11, 2025
Member
In that case, I think we need to detect the dind environment more precisely. I see you have |
crazy-max
commented
Aug 12, 2025
MemberAuthor
It's not really detecting the dind environment. When we create the container builder and request gpus: buildx/driver/docker-container/driver.go Lines 161 to 163 in c6d1e39 Moby will run the nvidia container runtime prestart hook that checks whether NVIDIA devices are requested and ensures GPU access is configured using the In such case, necessary libraries are mounted and GPU is detected with cc @elezar |
elezar
commented
Aug 12, 2025
As a naive question: Some of the output above referred to @crazy-max in the WSL case, where is the "outer" CDI spec being created (or should it be created)? Is this in the WSL2 distribution? Could a spec be generated For Docker Desktop as the VM is being started? On a related note, I have created moby/moby#50228 as a placeholder for migrating the |
crazy-max
commented
Aug 13, 2025
MemberAuthor
Yes we have the device name format defined here: https://docs.docker.com/reference/dockerfile/#run---device:
We don't handle CDI specs from the host for the container builder case atm. We use the
Yes good point, that might be something Docker Desktop could do.
Ah that looks ideal, I see the |
tonistiigi
approved these changes
Aug 13, 2025
Uh oh!
There was an error while loading. Please reload this page.
3 tasks
aevesdocker pushed a commit
to docker/docs
that referenced
this pull request
Aug 26, 2025
follow-up #23069 (comment) ## Related issues or tickets * docker/buildx#3320 ## Reviews <!-- Notes for reviewers here --> <!-- List applicable reviews (optionally @tag reviewers) --> - [ ] Technical review - [ ] Editorial review - [ ] Product review Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
relates to #3295 (comment)
When generating the CDI spec during build for a container builder created on WSL it fails with:
It seems requesting GPU on container builder creation introduced in #3063 is not enough as some libraries are missing on WSL.
To fix this issue we can mount the WSL libraries in the container if docker context is local.
Inspecting the CDI spec we can see
libdxcore.sois properly detected and mounted: