Mempool memory resource - IPC by keenan-simpson · Pull Request #446 · NVIDIA/cuda-python

keenan-simpson · 2025-02-12T19:30:10Z

This is ready for at least a first look

Add Mempool class as the first public MemoryResource implementation. It supports IPC.

Add IPC buffer which is an internal buffer implementation used to represent a buffer descriptor which is used to export and import buffers across process boundaries. It implements reduce which is expected by common multiprocessing libraries. It's use is seen in the tests added to this review.

copy-pr-bot · 2025-02-12T19:30:15Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

keenan-simpson · 2025-02-12T19:30:50Z

/ok to test

keenan-simpson · 2025-02-15T00:14:50Z

/ok to test

github-actions · 2025-02-15T00:35:39Z

Doc Preview CI
🚀 View preview at https://nvidia.github.io/cuda-python/pr-preview/pr-446/
https://nvidia.github.io/cuda-python/pr-preview/pr-446/cuda-core/
https://nvidia.github.io/cuda-python/pr-preview/pr-446/cuda-bindings/
Preview will be ready when the GitHub Pages deployment is complete.

keenan-simpson · 2025-02-24T18:42:26Z

/ok to test

keenan-simpson · 2025-02-24T19:09:43Z

/ok to test

keenan-simpson · 2025-02-24T19:38:55Z

/ok to test

keenan-simpson · 2025-02-24T23:06:24Z

/ok to test

keenan-simpson · 2025-02-25T17:27:05Z

/ok to test

keenan-simpson · 2025-02-25T18:15:06Z

/ok to test

keenan-simpson · 2025-02-25T18:37:58Z

/ok to test

kkraus14 · 2025-02-28T14:43:23Z

+# Check if pywin32 is available on Windows
+_PYWIN32_AVAILABLE = False
+if platform.system() == "Windows":
+ try:
+ import win32security
+
+ _PYWIN32_AVAILABLE = True
+ except ImportError:
+ import warnings
+
+ warnings.warn(
+ "pywin32 module not found. For better IPC support on Windows, " "install it with: pip install pywin32",
+ stacklevel=2,
+ )


How hard would it be to recreate the functionality we need from here to avoid this optional dependency?

I think we'll need to discuss our next move first and then clean this up based on the decision. Keenan struggled for quite a long time and it turns out the IPC support for mempools is not enabled on Windows at all. There is a number of doc and sample bugs we'll create later (the memory team created one).

I've created 2 bugs to address the documentation issues.
yeah the review is in limbo right now. I'm not sure if we should push the change for IPC mempools and have it only support linux for now, or if we should shelve it and add a cuMem and VMM API IPC memory_resource in the meantime.
All of the Windows specific changes should be treated as draft. That optional dependency is from when I was debugging windows errors and operating under the assumption that I was improperly creating the windows security descriptor for the mempool handle. In hindsight it was a (temporary) lack of suport in driver layer as Leo said.

keenan-simpson · 2025-03-05T18:13:20Z

/ok to test

keenan-simpson · 2025-03-06T17:25:21Z

I am tasked with adding the IPC mempool support for windows at the driver level (5151668), so I think we should roll with this implementation and have it pick up windows once that change is integrated. Users who need IPC on windows for older CTKs can implement their own MR or use the cuMemCreate & VMM API bindings. WDYT

leofang · 2025-09-17T23:05:04Z

IPC support for Linux landed (#930) 🎉 We'll revisit (and close) this PR when there's a viable path emerging for Windows counterpart (and preferably not through VMM #968) .

Removed preview folders for the following PRs: - PR #1305 - PR #446

leofang assigned keenan-simpson Feb 14, 2025

leofang added triage Needs the team's attention P0 High priority - Must do! feature New feature or request cuda.core Everything related to the cuda.core module labels Feb 14, 2025

keenan-simpson changed the title ~~WIP: IPC~~Mempool memory resource - IPCFeb 24, 2025

jrhemstad reviewed Feb 24, 2025
View reviewed changes

Comment threadcuda_core/cuda/core/experimental/_memory.py Outdated

add IPC support to default async mempool
247af8c

keenan-simpson force-pushed the IPC branch from 9fb8ff5 to 247af8cCompare February 24, 2025 22:45

keenan-simpson added 3 commits February 24, 2025 14:47

revert utils
4b52151

revert utils
050e4d6

rever utils
1da30fa

leofang self-requested a review February 25, 2025 05:50

fix typo
ebef438

keenan-simpson added 2 commits February 25, 2025 10:13

fix typo
5023339

merge main
d0f0bbc

another typo
e4da633

keenan-simpson added 2 commits February 25, 2025 13:40

make IPC buffer a descriptor not a buffer
77678e9

support windows
1711f78

keenan-simpson added 6 commits February 26, 2025 13:15

use library
2e5013a

use library
0c82df8

use library
1c6ee59

use library
3074742

use library
f35cf01

use library
bff995a

kkraus14 reviewed Feb 28, 2025
View reviewed changes

remove windows stuff from async mempool until it is supported
5fd7f46

keenan-simpson marked this pull request as ready for review March 5, 2025 18:13

Merge branch 'main' into IPC
d486fcb

keenan-simpson added 3 commits March 5, 2025 11:42

remove windows stuff from async mempool until it is supported
c39d9a9

Merge remote-tracking branch 'origin/IPC' into IPC
140ba3f

Merge branch 'main' into IPC
1bcebd5

leofang marked this pull request as draft May 2, 2025 04:30

This was referenced Jun 25, 2025
Ensure correct handling of buffers allocated with LegacyPinnedMemoryResource.allocate as kernel parameters #717
Merged
Ensure the mempool wrapped behind DeviceMemoryResource can be customized #727
Closed

leofang linked an issue Jun 26, 2025 that may be closed by this pull request
Ensure the mempool wrapped behind DeviceMemoryResource can be customized #727
Closed

leofang assigned Andy-Jost Aug 25, 2025

leofang added this to the cuda.core beta 7 milestone Aug 26, 2025

leofang removed the triage Needs the team's attention label Sep 11, 2025

leofang modified the milestones: cuda.core beta 7, cuda.core parking lot Sep 17, 2025

leofang added blocked This task is currently blocked by other tasks and removed P0 High priority - Must do! labels Sep 17, 2025

leofang mentioned this pull request Nov 20, 2025
Allow win32_handle_metadata to be an arbitrary object? #1263
Closed

Andy-Jost closed this Dec 4, 2025

github-actionsBot pushed a commit that referenced this pull request Dec 4, 2025
Clean up PR preview folders for 2 closed/merged PRs
ed66f59
Removed preview folders for the following PRs: - PR #1305 - PR #446

Uh oh!

Conversation

keenan-simpson commented Feb 12, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

copy-pr-botBot commented Feb 12, 2025

Uh oh!

keenan-simpson commented Feb 12, 2025

Uh oh!

keenan-simpson commented Feb 15, 2025

Uh oh!

github-actionsBot commented Feb 15, 2025

Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

keenan-simpson commented Feb 24, 2025

Uh oh!

keenan-simpson commented Feb 24, 2025

Uh oh!

keenan-simpson commented Feb 24, 2025

Uh oh!

Uh oh!

keenan-simpson commented Feb 24, 2025

Uh oh!

keenan-simpson commented Feb 25, 2025

Uh oh!

keenan-simpson commented Feb 25, 2025

Uh oh!

keenan-simpson commented Feb 25, 2025

Uh oh!

kkraus14Feb 28, 2025

Choose a reason for hiding this comment

Uh oh!

leofangMar 3, 2025

Choose a reason for hiding this comment

Uh oh!

keenan-simpsonMar 5, 2025

Choose a reason for hiding this comment

Uh oh!

keenan-simpson commented Mar 5, 2025

Uh oh!

keenan-simpson commented Mar 6, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

leofang commented Sep 17, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

keenan-simpson commented Feb 12, 2025•
edited
Loading

keenan-simpson commented Mar 6, 2025•
edited
Loading

leofang commented Sep 17, 2025•
edited
Loading