Overhaul of dpctl documentation by oleksandr-pavlyk · Pull Request #1619 · IntelPython/dpctl

oleksandr-pavlyk · 2024-03-28T15:00:39Z

This PR is a rewrite of dpctl's documentation. Here is the link to rendered docs based on sources in this PR.

It introduces Beginner's guide, User Guide, Reference Guide and contributor's guide
It does away with generate_rst.py and AutoAutosummary class, but uses Sphinx's autosummary extension to generate individual pages for classes, and array API functions.
It changes theme from read-the-docs to Furo
It modifies docstrings to fix sphinx issues
It modifies array API function signatures by using positional-only, keyword-only delimiters as done in the standard.
This required test changes and code changes.

Have you provided a meaningful PR description?
Have you added a test, reproducer or referred to an issue with a reproducer?
Have you tested your changes locally for CPU and GPU devices?
Have you made sure that new changes do not introduce compiler warnings?
Have you checked performance impact of proposed changes?
If this PR is a work in progress, are you opening the PR as a draft?

diptorupd · 2024-03-28T15:15:35Z

@oleksandr-pavlyk the PR will need to be split up. I suggest all docstring changes to be merged separately.

github-actions · 2024-03-28T15:39:47Z

Deleted rendered PR docs from intelpython.github.com/dpctl, latest should be updated shortly. 🤞

github-actions · 2024-03-28T15:41:13Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_181 ran successfully.
Passed: 838
Failed: 1
Skipped: 92

github-actions · 2024-03-28T15:57:42Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_181 ran successfully.
Passed: 838
Failed: 1
Skipped: 92

github-actions · 2024-03-28T16:33:32Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_181 ran successfully.
Passed: 838
Failed: 1
Skipped: 92

coveralls · 2024-03-28T16:56:55Z

coverage: 88.049%. remained the same
when pulling 1abd650 on docstring-edits
into 10b831c on master.

github-actions · 2024-03-28T17:03:53Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_182 ran successfully.
Passed: 838
Failed: 1
Skipped: 92

github-actions · 2024-03-28T19:04:56Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_186 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-03-28T19:45:22Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_187 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-03-28T21:00:56Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_189 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-03-31T03:01:34Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_191 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

oleksandr-pavlyk · 2024-04-01T13:24:07Z

@oleksandr-pavlyk the PR will need to be split up. I suggest all docstring changes to be merged separately.

I split the PR into gh-1625, and remaining docs-only changes in this PR opened into branch behind gh-1625.

github-actions · 2024-04-01T13:57:33Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_190 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-04-01T14:01:49Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_191 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-04-01T18:03:05Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_192 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-04-02T01:40:51Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_199 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-04-02T02:10:20Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_200 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-04-02T13:34:16Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_201 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-04-02T16:15:08Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_202 ran successfully.
Passed: 839
Failed: 0
Skipped: 92

github-actions · 2024-04-16T15:41:42Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_285 ran successfully.
Passed: 870
Failed: 8
Skipped: 92

ndgrigorian · 2024-04-16T22:13:33Z

+
+.. note::
+ Two instances :class:`dpctl.SyclQueue` may target the same ``sycl::device`` and be using the same ``sycl::context``, but correspond
+ to different scheduling enties, and hence be in violation of the compute-follows-data requirement. One common example of this are


Nit, but was this meant to say "entries"?

ndgrigorian · 2024-04-16T22:17:55Z

+If assignign USM-type "device" a score of 0, USM-type "shared" a score of 1, and USM-type "host" a score of 2,
+the USM-type of the output array has the smallest score of all its inputs.
+
+.. currentmodule:: dpctl.utils
+
+The convenience function :py:func:`get_coerced_usm_type` is a convenience function to determine the USM-type
+following this convention:


Suggested change
If assignign USM-type "device" a score of 0, USM-type "shared" a score of 1, and USM-type "host" a score of 2,
the USM-type of the output array has the smallest score of all its inputs.
.. currentmodule:: dpctl.utils
The convenience function :py:func:`get_coerced_usm_type` is a convenience function to determine the USM-type
following this convention:
If assigning USM-type "device" a score of 0, USM-type "shared" a score of 1, and USM-type "host" a score of 2,
the USM-type of the output array has the smallest score of all its inputs.
.. currentmodule:: dpctl.utils
The convenience function :py:func:`get_coerced_usm_type` determines the USM-type following this convention:

ndgrigorian · 2024-04-16T22:21:21Z

+ # r3 has value "host"
+ r3 = get_coerced_usm_type(["host", "host", "host"])
+
+Sharing data between devices and Python


It may be worthwhile to add an example demonstrating what happens when migrating a Numpy array of a data type not supported by the destination device.
Can be done later though.

ndgrigorian · 2024-04-16T22:23:14Z

+-------------
+
+* History of ``"dpctl"`` :ref:`name <beginners_guide_why_dpctl>`
+* Frequenty asked questions


Suggested change
* Frequenty asked questions
* Frequently asked questions

ndgrigorian · 2024-04-16T22:25:33Z

@@ -0,0 +1,69 @@
+.. _beginners_guide_why_dpctl:
+
+History of ``dpctl`` name


I wonder if this page should either be renamed or moved to the beginner's guides index page in the future. It stands out when looking at the list.

ndgrigorian · 2024-04-16T22:28:30Z

+ :class:`dpctl.tensor.usm_ndarray` objects one associated with ``q1`` and another associated with ``q2``
+ could not be combined in a call to the same function that implementes
+ :ref:`compute-followed-data programming model <dpctl_tensor_compute_follows_data>` in :mod:`dpctl.tensor`.


Suggested change
:class:`dpctl.tensor.usm_ndarray` objects one associated with ``q1`` and another associated with ``q2``
could not be combined in a call to the same function that implementes
:ref:`compute-followed-data programming model <dpctl_tensor_compute_follows_data>` in :mod:`dpctl.tensor`.
:class:`dpctl.tensor.usm_ndarray` objects one associated with ``q1`` and another associated with ``q2``
could not be combined in a call to the same function that implements
:ref:`compute-follows-data programming model <dpctl_tensor_compute_follows_data>` in :mod:`dpctl.tensor`.

ndgrigorian · 2024-04-16T22:34:32Z

+ x1 : dpctl.tensor.usm_ndarray,
+ x2 : dpctl.tensor.usm_ndarray
+ ):
+ exec_q = dpctil.utils.get_execution_queue((x1.sycl_queue, x2.sycl_queue))


Suggested change
exec_q = dpctil.utils.get_execution_queue((x1.sycl_queue, x2.sycl_queue))
exec_q = dpctl.utils.get_execution_queue((x1.sycl_queue, x2.sycl_queue))

ndgrigorian · 2024-04-16T22:35:53Z

+ ...
+
+In order to ensure that compute-follows-data works seemlessly out-of-the-box, :mod:`dpctl` maintains
+a cache of with context and device as keys and queues as values used by :class:`dpctl.tensor.Device` class.


Suggested change
a cache of with context and device as keys and queues as values used by :class:`dpctl.tensor.Device` class.
a cache with context and device as keys and queues as values used by :class:`dpctl.tensor.Device` class.

ndgrigorian · 2024-04-16T22:43:21Z

+ const auto &all_root_devs = sycl::device::get_devices();
+ auto beg = std::begin(all_root_devs);
+ auto end = std::end(all_root_devs);
+ auto selectot_fn = [parent_root_device](const sycl::device &root_d) -> bool {


Suggested change
auto selectot_fn = [parent_root_device](const sycl::device &root_d) -> bool {
auto selector_fn = [parent_root_device](const sycl::device &root_d) -> bool {

ndgrigorian · 2024-04-16T22:51:54Z

+ select_cpu_device
+ select_gpu_device
+ select_accelerator_device
+ select_device_with_aspects


This function could use a proper docstring later.

This was referring specifically to select_device_with_aspects, which has a description that isn't a true docstring (no real info about what types are permitted for a required_aspect, for instance).

ndgrigorian · 2024-04-16T23:03:58Z

+ SyclProgram
+ SyclKernel


Empty __init__() methods are picked up in the documentation of these, and the methods don't have pages or proper docstrings.

ndgrigorian · 2024-04-16T23:24:31Z

+:py:mod:`dpctl.tensor` is written using C++ and :sycl_spec_2020:`SYCL <>`
+and oneAPI extensions implemented in :dpcpp_compiler:`Intel(R) oneAPI DPC++ compiler <>`.
+
+This module contains:


Some undocumented things:
~~all constants (pi, e, nan, inf, and newaxis)~~
asnumpy and to_numpy
dtype itself
in elementwise functions, no documentation is rendered for nin, nout, types, etc. special attributes
print utilities do not have rendered documentation
~~flags object on usm_ndarray is mentioned under usm_ndarray.flags but has no rendered documentation~~
order keyword in general/memory layouts (C-contiguous vs. F-contiguous) may be worth a quick note

ndgrigorian · 2024-04-16T23:37:26Z

@oleksandr-pavlyk
In the future it may also be nice to change the libsyclinterface documentation rendering so that the global functions and so on are listed directly under the "on this page" section, with shortcuts. That way you don't have to scroll and click on them in the overview.

Some found in review, some identified by spell-checker tool.

…look at

Also added few workd to the list is known-to-be-correctly-spelled

github-actions · 2024-04-17T15:48:46Z

Array API standard conformance tests for dpctl=0.17.0dev0=py310h15de555_291 ran successfully.
Passed: 871
Failed: 7
Skipped: 92

ndgrigorian

Changes requested in my review have now either been committed or will be deferred to a later PR.

This LGTM, thank you @oleksandr-pavlyk! Very exciting to see these documentation improvements finally go in.

diptorupd

Let's merge it as-is for now, the changes in the PR are already a big improvement over status quo.

Also, the PR has become too large and all further change should be done incrementally.

oleksandr-pavlyk force-pushed the docstring-edits branch from f58650f to 0567168Compare March 28, 2024 15:19

oleksandr-pavlyk force-pushed the docstring-edits branch from 0567168 to 2348853Compare March 28, 2024 15:59

oleksandr-pavlyk marked this pull request as draft March 28, 2024 17:14

oleksandr-pavlyk mentioned this pull request Apr 1, 2024
Docstring edits only changes in documentation overhaul #1626
Merged
6 tasks

oleksandr-pavlyk force-pushed the docstring-edits branch from a597ccc to 4da37bcCompare April 1, 2024 13:23

oleksandr-pavlyk changed the base branch from master to docstring-edits-onlyApril 1, 2024 13:23

oleksandr-pavlyk force-pushed the docstring-edits-only branch from 891a1c5 to 7ac9ff2Compare April 1, 2024 17:25

oleksandr-pavlyk force-pushed the docstring-edits branch from 4840c5f to 6ef7cc5Compare April 1, 2024 17:25

vtavana reviewed Apr 1, 2024
View reviewed changes

Comment threaddocs/doc_sources/beginners_guides/index.rst
Comment threaddocs/doc_sources/beginners_guides/parallelisms.rst Outdated
Comment threaddocs/doc_sources/beginners_guides/device_info.rst Outdated
Comment threaddocs/doc_sources/beginners_guides/environment_variables.rst Outdated

vtavana reviewed Apr 1, 2024
View reviewed changes

Comment threaddocs/doc_sources/beginners_guides/environment_variables.rst Outdated

oleksandr-pavlyk force-pushed the docstring-edits branch from 4463a84 to 9bc0a86Compare April 2, 2024 01:36

ndgrigorian reviewed Apr 16, 2024
View reviewed changes

oleksandr-pavlyk changed the title ~~WIP: overhaul of dpctl documentation~~Overhaul of dpctl documentationApr 17, 2024

oleksandr-pavlyk added 6 commits April 17, 2024 08:38

Using sphinxcontrib.spelling as extension
ee22e70

Spelling fixes
175eba8
Some found in review, some identified by spell-checker tool.

Reworded no longer accurate statement that spell-checker prompted to …
a1093d9
…look at

Typos found by @ndigrigorian
b7d9515
Also added few workd to the list is known-to-be-correctly-spelled

Writeable->writable
b9fb29f

Fixed typo, added more known-as-spelled-correctly words
1abd650

ndgrigorian approved these changes Apr 17, 2024
View reviewed changes

diptorupd approved these changes Apr 17, 2024
View reviewed changes

oleksandr-pavlyk merged commit 182aede into masterApr 17, 2024

oleksandr-pavlyk deleted the docstring-edits branch April 17, 2024 19:10

ndgrigorian mentioned this pull request Apr 29, 2024
Render documentation for elementwise functions #1297
Closed

oleksandr-pavlyk mentioned this pull request Jul 19, 2024
Removed use of deprecated SYCL_DEVICE_FILTER #1740
Merged
6 tasks

ndgrigorian mentioned this pull request Feb 27, 2025
Add get_devices method to dpctl.SyclPlatform class #1992
Merged
8 tasks

ndgrigorian mentioned this pull request Apr 9, 2025
Update docs/requirements.txt#2045
Open

		@@ -0,0 +1,69 @@
		.. _beginners_guide_why_dpctl:

		History of ``dpctl`` name

	exec_q = dpctil.utils.get_execution_queue((x1.sycl_queue, x2.sycl_queue))
	exec_q = dpctl.utils.get_execution_queue((x1.sycl_queue, x2.sycl_queue))

	a cache of with context and device as keys and queues as values used by :class:`dpctl.tensor.Device` class.
	a cache with context and device as keys and queues as values used by :class:`dpctl.tensor.Device` class.

	auto selectot_fn = [parent_root_device](const sycl::device &root_d) -> bool {
	auto selector_fn = [parent_root_device](const sycl::device &root_d) -> bool {

Uh oh!

Conversation

oleksandr-pavlyk commented Mar 28, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

diptorupd commented Mar 28, 2024

Uh oh!

github-actionsBot commented Mar 28, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot commented Mar 28, 2024

Uh oh!

github-actionsBot commented Mar 28, 2024

Uh oh!

github-actionsBot commented Mar 28, 2024

Uh oh!

coveralls commented Mar 28, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot commented Mar 28, 2024

Uh oh!

github-actionsBot commented Mar 28, 2024

Uh oh!

github-actionsBot commented Mar 28, 2024

Uh oh!

github-actionsBot commented Mar 28, 2024

Uh oh!

github-actionsBot commented Mar 31, 2024

Uh oh!

oleksandr-pavlyk commented Apr 1, 2024

Uh oh!

github-actionsBot commented Apr 1, 2024

Uh oh!

github-actionsBot commented Apr 1, 2024

Uh oh!

github-actionsBot commented Apr 1, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actionsBot commented Apr 2, 2024

Uh oh!

github-actionsBot commented Apr 2, 2024

Uh oh!

github-actionsBot commented Apr 2, 2024

Uh oh!

github-actionsBot commented Apr 2, 2024

Uh oh!

github-actionsBot commented Apr 16, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ndgrigorianApr 16, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ndgrigorianApr 16, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

oleksandr-pavlyk commented Mar 28, 2024•
edited
Loading

github-actionsBot commented Mar 28, 2024•
edited
Loading

coveralls commented Mar 28, 2024•
edited
Loading

ndgrigorianApr 16, 2024•
edited
Loading

ndgrigorianApr 16, 2024•
edited
Loading

ndgrigorian commented Apr 16, 2024•
edited
Loading