Binary incompatibility with Raysect when building with numpy 2.0

[vsnever]

NumPy 2.0 was recently released. Although Cherab builds without errors with NumPy 2.0, trying to run any tests results in errors such as:

ImportError: Failed to import test module: cherab.core
Traceback (most recent call last):
 File "/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/unittest/loader.py", line 470, in _find_test_path
 package = self._get_module_from_name(name)
 File "/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
 __import__(name)
 File "/home/runner/work/cherab-core/cherab-core/cherab/core/__init__.py", line 20, in <module>
 from .atomic import *
 File "/home/runner/work/cherab-core/cherab-core/cherab/core/atomic/__init__.py", line 22, in <module>
 from .interface import AtomicData
 File "cherab/core/atomic/interface.pyx", line 1, in init cherab.core.atomic.interface
 # Copyright 2016-2022 Euratom
 File "cherab/core/atomic/zeeman.pyx", line 1, in init cherab.core.atomic.zeeman
 # Copyright 2016-2018 Euratom
 File "/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/raysect/core/__init__.py", line 30, in <module>
 from .ray import *
 File "raysect/core/ray.pyx", line 1, in init raysect.core.ray
 File "/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/raysect/core/math/__init__.py", line 39, in <module>
 from .statsarray import StatsBin, StatsArray1D, StatsArray2D, StatsArray3D
 File "raysect/core/math/statsarray.pyx", line 1, in init raysect.core.math.statsarray
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

Looks like NumPy 2.0 is binary incompatible with NumPy 1.x, which Raysect is built against. Until we have Raysect binaries built with NumPy 2.0, I suggest limiting the NumPy version to 1.x in setup.py, requirements.txt and ci.yml.

[jacklovell]

Yeah, this is sensible. We build Cherab wheels for PyPI using Numpy 1.x too (using the now-deprecated oldest-supported-numpy package), so will have the same problem. We can switch to specifying numpy~=2 for Python 3.9 and later since Numpy 2.0 produces a backward-compatible build, but for 3.7 (which UKAEA is still stuck on) and 3.8 would still need oldest-supported-numpy.

[vsnever]

When the Raysect binaries for NumPy 2.0 become available, we'll need to make the following changes:

pyproject.toml:

[build-system]
requires = [
 "setuptools>=62.3",
 "oldest-supported-numpy; python_version < '3.9'",
 "numpy~=2.0; python_version >= '3.9'",
 "cython~=3.0"
 "raysect==0.8.1",
]
build-backend="setuptools.build_meta"

requirements.txt:

cython~=3.0
numpy>=1.14,<2.0; python_version < '3.9'
numpy~=2.0; python_version >= '3.9'
scipy
matplotlib
raysect==0.8.1

setup.py:

 install_requires=[
 "cython~=3.0",
 "numpy>=1.14,<2.0; python_version < '3.9",
 "numpy~=2.0; python_version >= '3.9'"
 "scipy",
 "matplotlib",
 "raysect==0.8.1",
 ],

[munechika-koyo]

I found the incompatibility with numpy~=2.0 in the test script:

core/cherab/tools/tests/test_voxels.py

Lines 283 to 284 in 26595cb

	dx = coords[:, 0].ptp()
	dy = coords[:, 1].ptp()

.ptp() method seems to be deprecated, so it should be changed like:

dx, dy = np.ptp(coords, axis=0)

This change should not affect the package that depends on numpy<2.0, so would you mind if I apply the fix PR?

[munechika-koyo]

setup.py:

 install_requires=[
 "cython~=3.0",
 "numpy>=1.14,<2.0; python_version < '3.9",
 "numpy~=2.0; python_version >= '3.9'"
 "scipy",
 "matplotlib",
 "raysect==0.8.1",
 ],

Is it unnecessary to add cython for runtime?

[munechika-koyo]

This issue seems to be resolved by #486, so we should close it.