Cythonize away some perf hot spots

[leofang]

Description

Less aggressive version of #677.

Based on the summary in #658 (comment), this PR offers a performance optimization over identified hotspots to bring us to a lot closer with our reference (CuPy). The optimization strategy is to

• implement everything in pure Python (we do this today for cuda.core)
• once hotspots are identified, we lower to Cython
• most importantly, we still call cuda.bindingsPython APIs in the Cython code, so as to avoid introducing CTK as a build-time dependency (and therefore having to ship two separate packages cuda-core-cu11 and cuda-core-cu12)

In other words, this PR tries to find a reasonable balance between performance, easy of development, and easy of deployment, without introducing any breaking change.

Preliminary data:

• cuda.core main branch

In [5]: %timeite=dev.create_event()
4.65μs ± 18.4nsperloop (mean ± std. dev. of7runs, 100,000loopseach)

In [5]: %timeits=dev.create_stream()
7.7μs ± 9.98nsperloop (mean ± std. dev. of7runs, 100,000loopseach)

• this PR

In [6]: %timeite=dev.create_event()
1.11μs ± 6.91nsperloop (mean ± std. dev. of7runs, 1,000,000loopseach)

In [4]: %timeits=dev.create_stream()
4.12μs ± 14nsperloop (mean ± std. dev. of7runs, 100,000loopseach)

• cupy

In [8]: %timeite=cp.cuda.Event(disable_timing=True)
749ns ± 5.45nsperloop (mean ± std. dev. of7runs, 1,000,000loopseach)

In [14]: %timeits=cp.cuda.Stream(non_blocking=True)
3.8μs ± 8.54nsperloop (mean ± std. dev. of7runs, 100,000loopseach)

Checklist

• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[leofang]

/ok to test 227d9c1

[leofang]

/ok to test 48de1b3

[oleksandr-pavlyk]

/ok to test e95c4b1

[emcastillo]

LGTM!

[oleksandr-pavlyk]

/ok to test f4531e5

[oleksandr-pavlyk]

Since Event contains native class members, perhaps adding __cinit__ to initialize them is appropriate. Something like

 def __cinit__(self):
self._timing_disabled = False
self._busy_waited = False
self._device_id = -1

I also think it would be safe to set object class members to None.

This would ensure that Event.__new__(Event) would return an initialized struct.

[leofang]

I think Cython sets everything to None for us, but it'd be good to verify this indeed

Cython additionally takes responsibility of setting all object attributes to None,

https://cython.readthedocs.io/en/latest/src/userguide/special_methods.html#initialisation-methods-cinit-and-init

[oleksandr-pavlyk]

Ok, let's leave object members out. Should I push adding Event.__cinit__ ?

[leofang]

I think the same section says all members are zero/null initialized?

[oleksandr-pavlyk]

Yes, but is it appropriate to zero initialize _device_id? Perhaps it does not matter much.

[leofang]

CI is green

[oleksandr-pavlyk]

LGTM!

[github-actions]

Doc Preview CI
Preview removed because the pull request was closed or merged.