Indexing performance

[npolina4]

import dpctl.tensor as dpt
a = dpt.ones((8192, 8192), device='cpu', dtype='f4')
b = dpt.ones((8192, 8192), device='cpu', dtype=bool)
%timeit a[b]
#211 ms ± 6.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

import numpy
a_np = numpy.ones((8192, 8192), dtype='f4')
b_np = numpy.ones((8192, 8192), dtype=bool)
%timeit a_np[b_np]
#87.1 ms ± 2 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

[oleksandr-pavlyk]

This should be improved by changes in gh-1300. @npolina4 could you please post timeit results on the same machine you used to obtain reported numbers in the original comment?

[npolina4]

Result with changes in #1300
Size: 8192, 8192
numpy: 105 ms
cpu: 205 ms
gpu: 115 ms

Size: 4096, 4096
numpy: 24.5 ms
cpu: 45~80 ms
gpu: 21.4 ms

[oleksandr-pavlyk]

Using Core Ultra 7 155U laptop with integrated Arc GPU, and discrete NVidia GPU, and running on WSL (Ubuntu):

SYCL platform listings

[level_zero:gpu][level_zero:0] Intel(R) oneAPI Unified Runtime over Level-Zero, Intel(R) Graphics [0x7d45] 12.70.4 [1.6.31294.120000]
[opencl:cpu][opencl:0] Intel(R) OpenCL, Intel(R) Core(TM) Ultra 7 155U OpenCL 3.0 (Build 0) [2024.18.12.0.05_160000]
[opencl:gpu][opencl:1] Intel(R) OpenCL Graphics, Intel(R) Graphics [0x7d45] OpenCL 3.0 NEO [24.39.31294.12]
[cuda:gpu][cuda:0] NVIDIA CUDA BACKEND, NVIDIA GeForce RTX 3050 4GB Laptop GPU 8.6 [CUDA 12.6]

Scripts used for timing

# index.py
import dpctl.tensor as dpt
import time

n = 4096
shape = (n, n,)

a = dpt.ones(shape, dtype='f4')
m = dpt.ones(shape, dtype='b1')

reps = 54
t0 = time.perf_counter()
for _ in range(reps):
 r = a[m]
a.sycl_queue.wait()
t1 = time.perf_counter()

print((t1-t0)/reps)

# index_np.py
import numpy as np
import time

n = 4096
shape = (n, n,)

a = np.ones(shape, dtype='f4')
m = np.ones(shape, dtype='b1')

reps = 54
t0 = time.perf_counter()
for _ in range(reps):
 r = a[m]
t1 = time.perf_counter()

print((t1-t0)/reps)

I am getting about 16ms run-time for stock NumPy (version 2.2.1):

$ python -c "import numpy as np; print(np.__version__)"
2.2.1
$ python index_np.py
0.017485273816553806
$ for i in `seq 0 5`; do python index_np.py; done
0.022518442094291526
0.017075111110763694
0.015882405314456532
0.01638638164796349
0.015575324814697658
0.01593349300135203

Here is the summary of script runtimes per device:

Device	opencl:cpu	opencl:gpu	level_zero:gpu	cuda:gpu
Time, ms	38	14	25	9.5

Details

$ ONEAPI_DEVICE_SELECTOR=opencl:cpu python index.py
0.038533492259577744

$ ONEAPI_DEVICE_SELECTOR=opencl:gpu python index.py
0.014323775669456355

$ ONEAPI_DEVICE_SELECTOR=level_zero:gpu python index.py
0.02539329398078499

$ ONEAPI_DEVICE_SELECTOR=level_zero:gpu python index.py
0.02539329398078499

As such, I consider this issue resolved on the dpctl side.