gh-148675: Add Zd/Zf formats to array, ctypes, memoryview, struct

Doc/library/array.rst

@@ -52,6 +52,10 @@ defined:
 +-----------+--------------------+-------------------+-----------------------+-------+
 | ``'D'`` | double complex | complex | 16 | \(4) |
 +-----------+--------------------+-------------------+-----------------------+-------+
+| ``'Zf'`` | float complex | complex | 8 | \(4) |
++-----------+--------------------+-------------------+-----------------------+-------+
+| ``'Zd'`` | double complex | complex | 16 | \(4) |
++-----------+--------------------+-------------------+-----------------------+-------+
 
 
 Notes:
@@ -80,7 +84,7 @@ Notes:
  .. versionadded:: 3.15
 
 (4)
- Complex types (``F``and ``D``) are available unconditionally,
+ Complex types (``F``, ``D``, ``Zf`` and ``Zd``) are available unconditionally,
  regardless on support for complex types (the Annex G of the C11 standard)
  by the C compiler.
  As specified in the C11 standard, each complex type is represented by a
@@ -105,7 +109,10 @@ The module defines the following item:
 
 .. data:: typecodes
 
- A string with all available type codes.
+ A tuple with all available type codes.
+
+ .. versionchanged:: next
+ The type changed from :class:`str` to :class:`tuple`.
 
 
 The module defines the following type:

Doc/library/ctypes.rst

@@ -370,15 +370,19 @@ in both C and ``libffi``, the following complex types are available:
  * - :class:`c_float_complex`
  - :c:expr:`float complex`
  - :py:class:`complex`
- - ``'F'``
+ - ``'Zf'``
  * - :class:`c_double_complex`
  - :c:expr:`double complex`
  - :py:class:`complex`
- - ``'D'``
+ - ``'Zd'``
  * - :class:`c_longdouble_complex`
  - :c:expr:`long double complex`
  - :py:class:`complex`
- - ``'G'``
+ - ``'Zg'``
+
+.. versionchanged:: next
+ The :py:attr:`~_SimpleCData._type_` types ``F``, ``D`` and ``G`` have been
+ replaced with ``Zf``, ``Zd`` and ``Zg``.
 
 
 All these types can be created by calling them with an optional initializer of

Doc/library/struct.rst

@@ -264,6 +264,10 @@ platform-dependent.
 +--------+--------------------------+--------------------+----------------+------------+
 | ``D`` | :c:expr:`double complex` | complex | 16 | \(10) |
 +--------+--------------------------+--------------------+----------------+------------+
+| ``Zf`` | :c:expr:`float complex` | complex | 8 | \(10) |
++--------+--------------------------+--------------------+----------------+------------+
+| ``Zd`` | :c:expr:`double complex` | complex | 16 | \(10) |
++--------+--------------------------+--------------------+----------------+------------+
 | ``s`` | :c:expr:`char[]` | bytes | | \(9) |
 +--------+--------------------------+--------------------+----------------+------------+
 | ``p`` | :c:expr:`char[]` | bytes | | \(8) |
@@ -280,6 +284,9 @@ platform-dependent.
 .. versionchanged:: 3.14
  Added support for the ``'F'`` and ``'D'`` formats.
 
+.. versionchanged:: next
+ Added support for the ``'Zf'`` and ``'Zd'`` formats.
+
 .. seealso::
 
  The :mod:`array` and :ref:`ctypes <ctypes-fundamental-data-types>` modules,
@@ -372,7 +379,7 @@ Notes:
  For the ``'F'`` and ``'D'`` format characters, the packed representation uses
  the IEEE 754 binary32 and binary64 format for components of the complex
  number, regardless of the floating-point format used by the platform.
- Note that complex types (``F`` and ``D``) are available unconditionally,
+ Note that complex types (``F``/``Zf`` and ``D``/``Zd``) are available unconditionally,
  despite complex types being an optional feature in C.
  As specified in the C11 standard, each complex type is represented by a
  two-element C array containing, respectively, the real and imaginary parts.

Doc/whatsnew/3.15.rst

@@ -667,9 +667,9 @@ Other language changes
  (Contributed by James Hilton-Balfe in :gh:`128335`.)
 
 * The class :class:`memoryview` now supports the :c:expr:`float complex` and
- :c:expr:`double complex` C types: formatting characters ``'F'`` and ``'D'``
- respectively.
- (Contributed by Sergey B Kirpichev in :gh:`146151`.)
+ :c:expr:`double complex` C types: formatting characters ``'F'``/``'Zf'``
+and ``'D'``/``'Zd'`` respectively.
+ (Contributed by Victor Stinner in :gh:`146151` and :gh:`148675`.)
 
 * Allow the *count* argument of :meth:`bytes.replace` to be a keyword.
  (Contributed by Stan Ulbrych in :gh:`147856`.)
@@ -720,13 +720,17 @@ array
 -----
 
 * Support the :c:expr:`float complex` and :c:expr:`double complex` C types:
- formatting characters ``'F'`` and ``'D'`` respectively.
- (Contributed by Sergey B Kirpichev in :gh:`146151`.)
+ formatting characters ``'F'``/``'Zf'`` and ``'D'``/``'Zd'`` respectively.
+ (Contributed by Victor Stinner in :gh:`146151` and :gh:`148675`.)
 
 * Support half-floats (16-bit IEEE 754 binary interchange format): formatting
  character ``'e'``.
  (Contributed by Sergey B Kirpichev in :gh:`146238`.)
 
+* The :data:`array.typecodes` type changed from :class:`str` to :class:`tuple`
+ to support type codes longer than 1 character (``Zf`` and ``Zd``).
+ (Contributed by Victor Stinner in :gh:`148675`.)
+
 
 ast
 ---
@@ -1717,6 +1721,12 @@ ctypes
  which has been deprecated since Python 3.13.
  (Contributed by Bénédikt Tran in :gh:`133866`.)
 
+* Change the :py:attr:`~ctypes._SimpleCData._type_` of
+ :class:`~ctypes.c_float_complex`, :class:`~ctypes.c_double_complex` and
+ :class:`~ctypes.c_longdouble_complex` from ``F``, ``D`` and ``G`` to ``Zf``,
+ ``Zd`` and ``Zg`` for compatibility with numpy.
+ (Contributed by Victor Stinner in :gh:`148675`.)
+
 
 datetime
 --------

Lib/ctypes/__init__.py

@@ -206,13 +206,13 @@ class c_longdouble(_SimpleCData):
 
 try:
  class c_double_complex(_SimpleCData):
- _type_ = "D"
+ _type_ = "Zd"
  _check_size(c_double_complex)
  class c_float_complex(_SimpleCData):
- _type_ = "F"
+ _type_ = "Zf"
  _check_size(c_float_complex)
  class c_longdouble_complex(_SimpleCData):
- _type_ = "G"
+ _type_ = "Zg"
 except AttributeError:
  pass
 

Lib/test/test_array.py

@@ -31,7 +31,10 @@ class ArraySubclassWithKwargs(array.array):
  def __init__(self, typecode, newarg=None):
  array.array.__init__(self)
 
-typecodes = 'uwbBhHiIlLfdqQFDe'
+typecodes = (
+ 'u', 'w', 'b', 'B', 'h', 'H', 'i', 'I', 'l', 'L',
+ 'f', 'd', 'q', 'Q', 'F', 'D', 'e', 'Zf', 'Zd')
+
 
 class MiscTest(unittest.TestCase):
 
@@ -42,8 +45,9 @@ def test_array_is_sequence(self):
  def test_bad_constructor(self):
  self.assertRaises(TypeError, array.array)
  self.assertRaises(TypeError, array.array, spam=42)
- self.assertRaises(TypeError, array.array, 'xx')
+ self.assertRaises(ValueError, array.array, 'xx')
  self.assertRaises(ValueError, array.array, 'x')
+ self.assertRaises(ValueError, array.array, 'Z')
 
  @support.cpython_only
  def test_disallow_instantiation(self):
@@ -85,6 +89,12 @@ def __index__(self):
  with self.assertRaises(TypeError):
  a.fromlist(lst)
 
+ def test_typecodes(self):
+ self.assertIsInstance(array.typecodes, tuple)
+ for typecode in array.typecodes:
+ self.assertIsInstance(typecode, str)
+ self.assertGreaterEqual(len(typecode), 1)
+
 
 # Machine format codes.
 #
@@ -208,6 +218,14 @@ def test_numbers(self):
  [9006104071832581.0j, float('inf'), complex('1-infj'), -0.0]),
  (['D'], IEEE_754_DOUBLE_COMPLEX_BE, '>DDDD',
  [9006104071832581.0j, float('inf'), complex('1-infj'), -0.0]),
+ (['Zf'], IEEE_754_FLOAT_COMPLEX_LE, '<ZfZfZfZf',
+ [16711938.0j, float('inf'), complex('1-infj'), -0.0]),
+ (['Zf'], IEEE_754_FLOAT_COMPLEX_BE, '>ZfZfZfZf',
+ [16711938.0j, float('inf'), complex('1-infj'), -0.0]),
+ (['Zd'], IEEE_754_DOUBLE_COMPLEX_LE, '<ZdZdZdZd',
+ [9006104071832581.0j, float('inf'), complex('1-infj'), -0.0]),
+ (['Zd'], IEEE_754_DOUBLE_COMPLEX_BE, '>ZdZdZdZd',
+ [9006104071832581.0j, float('inf'), complex('1-infj'), -0.0]),
  )
  for testcase in testcases:
  valid_typecodes, mformat_code, struct_fmt, values = testcase
@@ -1237,6 +1255,9 @@ def test_free_after_iterating(self):
  support.check_free_after_iterating(self, reversed, array.array,
  (self.typecode,))
 
+ def test_known_typecode(self):
+ self.assertIn(self.typecode, array.typecodes)
+
 class StringTest(BaseTest):
 
  def test_setitem(self):
@@ -1576,7 +1597,7 @@ def test_nan(self):
  def test_byteswap(self):
  a = array.array(self.typecode, self.example)
  self.assertRaises(TypeError, a.byteswap, 42)
- if a.itemsize in (1, 2, 4, 8):
+ if a.itemsize in (1, 2, 4, 8, 16):
  b = array.array(self.typecode, self.example)
  b.byteswap()
  if a.itemsize == 1:
@@ -1631,6 +1652,14 @@ class ComplexDoubleTest(CFPTest, unittest.TestCase):
  typecode = 'D'
  minitemsize = 16
 
+class ComplexZfFloatTest(CFPTest, unittest.TestCase):
+ typecode = 'Zf'
+ minitemsize = 8
+
+class ComplexZdDoubleTest(CFPTest, unittest.TestCase):
+ typecode = 'Zd'
+ minitemsize = 16
+
 
 class LargeArrayTest(unittest.TestCase):
  typecode = 'b'

Lib/test/test_buffer.py

@@ -67,7 +67,7 @@
  'h':0, 'H':0, 'i':0, 'I':0,
  'l':0, 'L':0, 'n':0, 'N':0,
  'e':0, 'f':0, 'd':0, 'P':0,
- 'F':0, 'D':0
+ 'F':0, 'D':0, 'Zf':0, 'Zd':0,
 }
 
 # NumPy does not have 'n' or 'N':
@@ -95,7 +95,9 @@
  'e':(-65519, 65520), 'f':(-(1<<63), 1<<63),
  'd':(-(1<<1023), 1<<1023),
  'F':(-(1<<63), 1<<63),
- 'D':(-(1<<1023), 1<<1023)
+ 'D':(-(1<<1023), 1<<1023),
+ 'Zf':(-(1<<63), 1<<63),
+ 'Zd':(-(1<<1023), 1<<1023),
 }
 
 def native_type_range(fmt):
@@ -110,9 +112,9 @@ def native_type_range(fmt):
  lh = (-(1<<63), 1<<63)
  elif fmt == 'd':
  lh = (-(1<<1023), 1<<1023)
- elif fmt == 'F':
+ elif fmt in ('F', 'Zf'):
  lh = (-(1<<63), 1<<63)
- elif fmt == 'D':
+ elif fmt in ('D', 'Zd'):
  lh = (-(1<<1023), 1<<1023)
  else:
  for exp in (128, 127, 64, 63, 32, 31, 16, 15, 8, 7):
@@ -182,18 +184,28 @@ def randrange_fmt(mode, char, obj):
  if char in 'efd':
  x = struct.pack(char, x)
  x = struct.unpack(char, x)[0]
- if char in 'FD':
+ if char in ('F', 'D', 'Zf', 'Zd'):
  y = randrange(*fmtdict[mode][char])
  x = complex(x, y)
  x = struct.pack(char, x)
  x = struct.unpack(char, x)[0]
  return x
 
+def split_format(fmt):
+ i = 0
+ while i < len(fmt):
+ if fmt[i] == 'Z':
+ n = 2
+ else:
+ n = 1
+ yield fmt[i:i + n]
+ i += n
+
 def gen_item(fmt, obj):
  """Return single random item."""
  mode, chars = fmt.split('#')
  x = []
- for c in chars:
+ for c in split_format(chars):
  x.append(randrange_fmt(mode, c, obj))
  return x[0] if len(x) == 1 else tuple(x)
 
@@ -254,9 +266,7 @@ def is_byte_format(fmt):
 
 def is_memoryview_format(fmt):
  """format suitable for memoryview"""
- x = len(fmt)
- return ((x == 1 or (x == 2 and fmt[0] == '@')) and
- fmt[x-1] in MEMORYVIEW)
+ return fmt.removeprefix('@') in MEMORYVIEW
 
 NON_BYTE_FORMAT = [c for c in fmtdict['@'] if not is_byte_format(c)]
 
@@ -648,14 +658,22 @@ def ndarray_from_structure(items, fmt, t, flags=0):
  return ndarray(items, shape=shape, strides=strides, format=fmt,
  offset=offset, flags=ND_WRITABLE|flags)
 
+# Convert PEP 3118 formats to numpy dtypes
+FORMAT_TO_DTYPE = {
+ 'Zf': 'F',
+ 'Zd': 'D',
+}
+
 def numpy_array_from_structure(items, fmt, t):
  """Return numpy_array from the tuple returned by rand_structure()"""
  memlen, itemsize, ndim, shape, strides, offset = t
  buf = bytearray(memlen)
  for j, v in enumerate(items):
  struct.pack_into(fmt, buf, j*itemsize, v)
+ # Replace Zd/Zf formats with D/F dtypes
+ dtype = FORMAT_TO_DTYPE.get(fmt, fmt)
  return numpy_array(buffer=buf, shape=shape, strides=strides,
- dtype=fmt, offset=offset)
+ dtype=dtype, offset=offset)
 
 
 # ======================================================================
@@ -3037,7 +3055,7 @@ def test_memoryview_assign(self):
  continue
  m2 = m1.cast(fmt)
  lo, hi = _range
- if fmt in "dfDF":
+ if fmt in ("d", "f", "D", "F", "Zd", "Zf"):
  lo, hi = -2**1024, 2**1024
  if fmt != 'P': # PyLong_AsVoidPtr() accepts negative numbers
  self.assertRaises(ValueError, m2.__setitem__, 0, lo-1)

Lib/test/test_ctypes/test_c_simple_type_meta.py

@@ -213,14 +213,12 @@ def test_bad_type_message(self):
  class F(metaclass=PyCSimpleType):
  _type_ = "\0"
  message = str(cm.exception)
- expected_type_chars = list('cbBhHiIlLdDFGfuzZqQPXOv?g')
- if not hasattr(ctypes, 'c_float_complex'):
- expected_type_chars.remove('F')
- expected_type_chars.remove('D')
- expected_type_chars.remove('G')
+ expected_type_chars = list('cbBhHiIlLdfuzZqQPXOv?g')
  if not MS_WINDOWS:
  expected_type_chars.remove('X')
  self.assertIn("'" + ''.join(expected_type_chars) + "'", message)
+ if hasattr(ctypes, 'c_float_complex'):
+ self.assertIn("'Zf', 'Zd', 'Zg'", message)
 
  def test_creating_pointer_in_dunder_init_3(self):
  """Check if interfcase subclasses properly creates according internal

Lib/test/test_ctypes/test_numbers.py

@@ -119,15 +119,24 @@ def test_floats(self):
  @unittest.skipUnless(hasattr(ctypes, "c_double_complex"),
  "requires C11 complex type")
  def test_complex(self):
- for t in [ctypes.c_double_complex, ctypes.c_float_complex,
- ctypes.c_longdouble_complex]:
+ for format, t in [
+ ('Zd', ctypes.c_double_complex),
+ ('Zf', ctypes.c_float_complex),
+ ('Zg', ctypes.c_longdouble_complex),
+ ]:
  self.assertEqual(t(1).value, 1+0j)
  self.assertEqual(t(1.0).value, 1+0j)
  self.assertEqual(t(1+0.125j).value, 1+0.125j)
  self.assertEqual(t(IndexLike()).value, 2+0j)
  self.assertEqual(t(FloatLike()).value, 2+0j)
  self.assertEqual(t(ComplexLike()).value, 1+1j)
 
+ prefix = '>' if sys.byteorder == 'big' else '<'
+ num = t(1.0)
+ self.assertEqual(memoryview(num).format, prefix + format)
+ array = (t * 3)()
+ self.assertEqual(memoryview(array).format, prefix + format)
+
  @unittest.skipUnless(hasattr(ctypes, "c_double_complex"),
  "requires C11 complex type")
  def test_complex_round_trip(self):