Skip to content

Add some basic test coverage for cuda.core.experimental - #153

Merged
keenan-simpson merged 13 commits into
mainfrom
ksimpson/cuda_core_tests
Oct 29, 2024
Merged

Add some basic test coverage for cuda.core.experimental#153
keenan-simpson merged 13 commits into
mainfrom
ksimpson/cuda_core_tests

Conversation

@keenan-simpson

@keenan-simpsonkeenan-simpson commented Oct 10, 2024

Copy link
Copy Markdown
Contributor

Close#78.

This change adds a minimal testsuite to the repo so we can establish pipeline, and develop the tests alongside the cuda.core object model. It uses a pytest tree structure.

Example:

python -m pytest tests/
===================================================== test session starts ======================================================
platform linux -- Python 3.12.7, pytest-8.3.3, pluggy-1.5.0
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/ksimpson/cuda-python/cuda-python/cuda_core
configfile: pyproject.toml
plugins: benchmark-4.0.0
collected 8 items
tests/test_context.py . [ 12%]
tests/test_device.py . [ 25%]
tests/test_event.py . [ 37%]
tests/test_launcher.py . [ 50%]
tests/test_memory.py . [ 62%]
tests/test_module.py . [ 75%]
tests/test_program.py . [ 87%]
tests/test_stream.py . [100%]
====================================================== 8 passed in 2.11s =======================================================

@copy-pr-bot

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@leofangleofang added P0 High priority - Must do! test Improvements or additions to tests cuda.core Everything related to the cuda.core module labels Oct 11, 2024
@leofangleofang added this to the cuda.core beta 1 milestone Oct 11, 2024
@m3vaz

Copy link
Copy Markdown
Collaborator

/ok to test

m3vaz
m3vaz previously approved these changes Oct 15, 2024
@leofang

Copy link
Copy Markdown
Member

Let's also hook up the code samples with the test system. We can do something like this: Collect the test files in-process, and loop over them.

Comment threadcuda_core/tests/test_compiler.py Outdated
Comment threadcuda_core/tests/test_context.py Outdated
@leofang

leofang commented Oct 15, 2024

Copy link
Copy Markdown
Member

@ksimpson-work heads up, we've done the move to the experimental submodule (#171), so you might need to merge with the latest main and make some changes 😅

@keenan-simpson
keenan-simpsonforce-pushed the ksimpson/cuda_core_tests branch from 1d5f425 to fc6d176CompareOctober 25, 2024 00:16
@keenan-simpsonkeenan-simpson changed the title Add barebones tests to repo to start a pipelineAdd some basic test coverage for cuda.core.experimentalOct 25, 2024
@leofang
leofang requested review from vzhurba01 and removed request for jollyliliOctober 25, 2024 18:20
Comment threadcuda_core/tests/test_device.py Outdated
Comment threadcuda_core/tests/test_device.py Outdated
Comment threadcuda_core/tests/test_device.py Outdated
Comment threadcuda_core/tests/test_device.py
Comment threadcuda_core/tests/test_examples.py Outdated
Comment threadcuda_core/tests/test_event.py Outdated
Comment threadcuda_core/tests/test_event.py Outdated
Comment threadcuda_core/tests/test_launcher.py Outdated
Comment threadcuda_core/tests/conftest.py
Comment threadcuda_core/tests/conftest.py
Comment threadcuda_core/tests/example_tests/utils.py
Comment threadcuda_core/tests/test_device.py
Comment threadcuda_core/tests/test_launcher.py
Comment threadcuda_core/tests/test_memory.py
Comment threadcuda_core/tests/test_program.py
Comment threadcuda_core/tests/test_stream.py
Comment threadcuda_core/tests/example_tests/utils.py Outdated
Comment threadcuda_core/tests/example_tests/utils.py Outdated
Comment threadcuda_core/tests/example_tests/utils.py
Comment threadcuda_core/tests/example_tests/utils.py Outdated
Comment threadcuda_core/tests/example_tests/test_basic_examples.py Outdated
Comment threadcuda_core/tests/example_tests/test_basic_examples.py Outdated

@vzhurba01vzhurba01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this review I'm done looking at all the files. I'm sorry for all the nits, and thank you for handling the ones I've already shared! 🙏

Comment threadcuda_core/tests/test_memory.py Outdated
Comment threadcuda_core/tests/test_memory.py Outdated
Comment threadcuda_core/tests/test_memory.py Outdated
Comment threadcuda_core/tests/conftest.py Outdated
Comment threadcuda_core/tests/test_memory.py Outdated
Comment threadcuda_core/tests/example_tests/utils.py
Comment threadcuda_core/tests/test_program.py
Comment threadcuda_core/tests/test_device.py
Comment threadcuda_core/tests/test_event.py
Comment threadcuda_core/tests/conftest.py Outdated
@leofang

Copy link
Copy Markdown
Member

Let's get the formatting feedbacks addressed quickly and move on. I've opened #195 to track an automated solution so that we don't need to spend time on reviewing formatting issues, which could sometimes be distracting/frustrating 🙂

@keenan-simpson

Copy link
Copy Markdown
ContributorAuthor

Thankyou @vzhurba01 for your review. I have addressed all the comments and implemented all your nits as I agreed with all. Thanks @leofang for your review as well. I believe all comments have been addressed as of the most recent push

vzhurba01
vzhurba01 previously approved these changes Oct 28, 2024

@vzhurba01vzhurba01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@leofangleofang left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Keenan/Vlad, for all the hard work! Unless I am missing something I don't think we have "relational tests" added yet (ex: Device.create_stream() returns a Stream; Stream.record() returns an Event, etc), but let's add them in the next PR.

@leofangleofang left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ksimpson-work do you see these warnings locally? This is concerning and making me wonder if we need to fix #141 as part of beta 1:

============================================================== warnings summary ===============================================================
<frozen importlib._bootstrap_external>:1297
<frozen importlib._bootstrap_external>:1297: DeprecationWarning: The cuda.cuda module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.driver module instead.
<frozen importlib._bootstrap_external>:1297
<frozen importlib._bootstrap_external>:1297: DeprecationWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
<frozen importlib._bootstrap_external>:1297
<frozen importlib._bootstrap_external>:1297: DeprecationWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.
tests/test_memory.py::test_buffer_initialization
tests/test_memory.py::test_buffer_copy_to
tests/test_memory.py::test_buffer_copy_from
/local/home/leof/miniforge3/envs/cuda_py_cuda126/lib/python3.12/site-packages/_pytest/unraisableexception.py:85: PytestUnraisableExceptionWarning: Exception ignored in: <function Buffer.__del__ at 0x7f3fb7ce1ee0>
Traceback (most recent call last):
File "/local/home/leof/dev/cuda-python/cuda_core/cuda/core/experimental/_memory.py", line 35, in __del__
self.close(default_stream())
File "/local/home/leof/dev/cuda-python/cuda_core/cuda/core/experimental/_memory.py", line 41, in close
self._mr.deallocate(self._ptr, self._size, stream)
File "/local/home/leof/dev/cuda-python/cuda_core/tests/test_memory.py", line 96, in deallocate
handle_return(cuda.cuMemFreeHost(ptr))
File "/local/home/leof/dev/cuda-python/cuda_core/cuda/core/experimental/_utils.py", line 57, in handle_return
_check_error(result[0], handle=handle)
File "/local/home/leof/dev/cuda-python/cuda_core/cuda/core/experimental/_utils.py", line 29, in _check_error
raise CUDAError(f"{name.decode()}: {desc.decode()}")
cuda.core.experimental._utils.CUDAError: CUDA_ERROR_INVALID_VALUE: invalid argument
warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
...

Could you check if explicitly calling Buffer.close() by the end of each test would make it go away? If so we should add it for now and circle back right after beta 1 (#141 needs a long-term solution).

@leofang

Copy link
Copy Markdown
Member

Could you check if explicitly calling Buffer.close() by the end of each test would make it go away?

Confirmed locally that this is the way to go. mr.deallocate was meant to be called by buf.close, not by end users.

Comment threadcuda_core/cuda/core/experimental/_program.py
@leofang

Copy link
Copy Markdown
Member

but let's add them in the next PR.

Tracked in #196. @vzhurba01 @ksimpson-work merge?

@leofang

Copy link
Copy Markdown
Member

@ksimpson-work feel free to merge!

@keenan-simpson
keenan-simpson merged commit 213baf4 into mainOct 29, 2024
@leofang
leofang deleted the ksimpson/cuda_core_tests branch October 29, 2024 00:53
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cuda.coreEverything related to the cuda.core moduleP0High priority - Must do!testImprovements or additions to tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add basic functionality tests for cuda.core

4 participants

@keenan-simpson@m3vaz@leofang@vzhurba01