gh-140550: PEP 793 reference documentation

Doc/c-api/extension-modules.rst

@@ -8,7 +8,8 @@ Defining extension modules
 A C extension for CPython is a shared library (for example, a ``.so`` file
 on Linux, ``.pyd`` DLL on Windows), which is loadable into the Python process
 (for example, it is compiled with compatible compiler settings), and which
-exports an :ref:`initialization function <extension-export-hook>`.
+exports an :dfn:`export hook` function (or an
+old-style :ref:`initialization function <extension-pyinit>`).
 
 To be importable by default (that is, by
 :py:class:`importlib.machinery.ExtensionFileLoader`),
@@ -23,25 +24,127 @@ and must be named after the module name plus an extension listed in
  One suitable tool is Setuptools, whose documentation can be found at
  https://setuptools.pypa.io/en/latest/setuptools.html.
 
-Normally, the initialization function returns a module definition initialized
-using :c:func:`PyModuleDef_Init`.
-This allows splitting the creation process into several phases:
+.. _extension-export-hook:
+
+Extension export hook
+.....................
+
+.. versionadded:: next
+
+ Support for the :samp:`PyModExport_{<name>}` export hook was added in Python
+ 3.15. The older way of defining modules is still available: consult either
+ the :ref:`extension-pyinit` section or earlier versions of this
+ documentation if you plan to support earlier Python versions.
+
+The export hook must be an exported function with the following signature:
+
+.. c:function:: PyModuleDef_Slot *PyModExport_modulename(void)
+
+For modules with ASCII-only names, the :ref:`export hook <extension-export-hook>`
+must be named :samp:`PyModExport_{<name>}`,
+with ``<name>`` replaced by the module's name.
+
+For non-ASCII module names, the export hook must instead be named
+:samp:`PyModExportU_{<name>}` (note the ``U``), with ``<name>`` encoded using
+Python's *punycode* encoding with hyphens replaced by underscores. In Python:
+
+.. code-block:: python
+
+ def hook_name(name):
+ try:
+ suffix = b'_' + name.encode('ascii')
+ except UnicodeEncodeError:
+ suffix = b'U_' + name.encode('punycode').replace(b'-', b'_')
+ return b'PyModExport' + suffix
+
+The export hook returns an array of :c:type:`PyModuleDef_Slot` entries,
+terminated by an entry with a slot ID of ``0``.
+These slots describe how the module should be created and initialized.
+
+This array must remain valid and constant until interpreter shutdown.
+Typically, it should use ``static`` storage.
+Prefer using the :c:macro:`Py_mod_create` and :c:macro:`Py_mod_exec` slots
+for any dynamic behavior.
+
+The export hook may return ``NULL`` with an exception set to signal failure.
+
+It is recommended to define the export hook function using a helper macro:
+
+.. c:macro:: PyMODEXPORT_FUNC
+
+ Declare an extension module export hook.
+ This macro:
+
+ * specifies the :c:expr:`PyModuleDef_Slot*` return type,
+ * adds any special linkage declarations required by the platform, and
+ * for C++, declares the function as ``extern "C"``.
 
+For example, a module called ``spam`` would be defined like this::
+
+ PyABIInfo_VAR(abi_info);
+
+ static PyModuleDef_Slot spam_slots[] = {
+ {Py_mod_abi, &abi_info},
+ {Py_mod_name, "spam"},
+ {Py_mod_init, spam_init_function},
+ ...
+ {0, NULL},
+ };
+
+ PyMODEXPORT_FUNC
+ PyModExport_spam(void)
+ {
+ return spam_slots;
+ }
+
+The export hook is typically the only non-\ ``static``
+item defined in the module's C source.
+
+The hook should be kept short -- ideally, one line as above.
+If you do need to use Python C API in this function, it is recommended to call
+``PyABIInfo_Check(&abi_info, "modulename")`` first to raise an exception,
+rather than crash, in common cases of ABI mismatch.
+
+
+.. note::
+
+ It is possible to export multiple modules from a single shared library by
+ defining multiple export hooks.
+ However, importing them requires a custom importer or suitably named
+ copies/links of the extension file, because Python's import machinery only
+ finds the function corresponding to the filename.
+ See the `Multiple modules in one library <https://peps.python.org/pep-0489/#multiple-modules-in-one-library>`__
+ section in :pep:`489` for details.
+
+
+.. _multi-phase-initialization:
+
+Multi-phase initialization
+..........................
+
+The process of creating an extension module follows several phases:
+
+- Python finds and calls the export hook to get information on how to
+ create the module.
 - Before any substantial code is executed, Python can determine which
  capabilities the module supports, and it can adjust the environment or
  refuse loading an incompatible extension.
-- By default, Python itself creates the module object -- that is, it does
- the equivalent of :py:meth:`object.__new__` for classes.
- It also sets initial attributes like :attr:`~module.__package__` and
- :attr:`~module.__loader__`.
-- Afterwards, the module object is initialized using extension-specific
- code -- the equivalent of :py:meth:`~object.__init__` on classes.
+ Slots like :c:data:`Py_mod_abi`, :c:data:`Py_mod_gil` and
+ :c:data:`Py_mod_multiple_interpreters` influence this step.
+- By default, Python itself then creates the module object -- that is, it does
+ the equivalent of calling :py:meth:`~object.__new__` when creating an object.
+ This step can be overridden using the :c:data:`Py_mod_create` slot.
+- Python sets initial module attributes like :attr:`~module.__package__` and
+ :attr:`~module.__loader__`, and inserts the module object into
+ :py:attr:`sys.modules`.
+- Afterwards, the module object is initialized in an extension-specific way
+ -- the equivalent of :py:meth:`~object.__init__` when creating an object,
+ or of executing top-level code in a Python-language module.
+ The behavior is specified using the :c:data:`Py_mod_exec` slot.
 
 This is called *multi-phase initialization* to distinguish it from the legacy
-(but still supported) *single-phase initialization* scheme,
-where the initialization function returns a fully constructed module.
-See the :ref:`single-phase-initialization section below <single-phase-initialization>`
-for details.
+(but still supported) :ref:`single-phase initialization <single-phase-initialization>`,
+where an initialization function returns a fully constructed module.
 
 .. versionchanged:: 3.5
 
@@ -53,7 +156,7 @@ Multiple module instances
 
 By default, extension modules are not singletons.
 For example, if the :py:attr:`sys.modules` entry is removed and the module
-is re-imported, a new module object is created, and typically populated with
+is re-imported, a new module object is created and, typically, populated with
 fresh method and type objects.
 The old module is subject to normal garbage collection.
 This mirrors the behavior of pure-Python modules.
@@ -83,36 +186,34 @@ A module may also be limited to the main interpreter using
 the :c:data:`Py_mod_multiple_interpreters` slot.
 
 
-.. _extension-export-hook:
+.. _extension-pyinit:
 
-Initialization function
-.......................
+``PyInit`` function
+...................
 
-The initialization function defined by an extension module has the
-following signature:
+.. deprecated:: next
+
+ This functionality is :term:`soft deprecated`.
+ It will not get new features, but there are no plans to remove it.
+
+Instead of :c:func:`PyModExport_modulename`, an extension module can define
+an older-style :dfn:`initialization function` with the signature:
 
 .. c:function:: PyObject* PyInit_modulename(void)
 
 Its name should be :samp:`PyInit_{<name>}`, with ``<name>`` replaced by the
 name of the module.
+For non-ASCII module names, use :samp:`PyInitU_{<name>}` instead, with
+``<name>`` encoded in the same way as for the
+:ref:`export hook <extension-export-hook>` (that is, using Punycode
+with underscores).
 
-For modules with ASCII-only names, the function must instead be named
-:samp:`PyInit_{<name>}`, with ``<name>`` replaced by the name of the module.
-When using :ref:`multi-phase-initialization`, non-ASCII module names
-are allowed. In this case, the initialization function name is
-:samp:`PyInitU_{<name>}`, with ``<name>`` encoded using Python's
-*punycode* encoding with hyphens replaced by underscores. In Python:
+If a module exports both :samp:`PyInit_{<name>}` and
+:samp:`PyModExport_{<name>}`, the :samp:`PyInit_{<name>}` function
+is ignored.
 
-.. code-block:: python
-
- def initfunc_name(name):
- try:
- suffix = b'_' + name.encode('ascii')
- except UnicodeEncodeError:
- suffix = b'U_' + name.encode('punycode').replace(b'-', b'_')
- return b'PyInit' + suffix
-
-It is recommended to define the initialization function using a helper macro:
+Like with :c:macro:`PyMODEXPORT_FUNC`, it is recommended to define the
+initialization function using a helper macro:
 
 .. c:macro:: PyMODINIT_FUNC
 
@@ -123,51 +224,24 @@ It is recommended to define the initialization function using a helper macro:
  * adds any special linkage declarations required by the platform, and
  * for C++, declares the function as ``extern "C"``.
 
-For example, a module called ``spam`` would be defined like this::
-
- static struct PyModuleDef spam_module = {
- .m_base = PyModuleDef_HEAD_INIT,
- .m_name = "spam",
- ...
- };
-
- PyMODINIT_FUNC
- PyInit_spam(void)
- {
- return PyModuleDef_Init(&spam_module);
- }
-
-It is possible to export multiple modules from a single shared library by
-defining multiple initialization functions. However, importing them requires
-using symbolic links or a custom importer, because by default only the
-function corresponding to the filename is found.
-See the `Multiple modules in one library <https://peps.python.org/pep-0489/#multiple-modules-in-one-library>`__
-section in :pep:`489` for details.
-
-The initialization function is typically the only non-\ ``static``
-item defined in the module's C source.
 
+Normally, the initialization function (``PyInit_modulename``) returns
+a :c:type:`PyModuleDef` instance with non-``NULL``
+:c:member:`~PyModuleDef.m_slots`. This allows Python to use
+:ref:`multi-phase initialization <multi-phase-initialization>`.
 
-.. _multi-phase-initialization:
-
-Multi-phase initialization
-..........................
-
-Normally, the :ref:`initialization function <extension-export-hook>`
-(``PyInit_modulename``) returns a :c:type:`PyModuleDef` instance with
-non-``NULL`` :c:member:`~PyModuleDef.m_slots`.
 Before it is returned, the ``PyModuleDef`` instance must be initialized
 using the following function:
 
-
 .. c:function:: PyObject* PyModuleDef_Init(PyModuleDef *def)
 
  Ensure a module definition is a properly initialized Python object that
  correctly reports its type and a reference count.
 
  Return *def* cast to ``PyObject*``, or ``NULL`` if an error occurred.
 
- Calling this function is required for :ref:`multi-phase-initialization`.
+ Calling this function is required before returning a :c:type:`PyModuleDef`
+ from a module initialization function.
  It should not be used in other contexts.
 
  Note that Python assumes that ``PyModuleDef`` structures are statically
@@ -178,18 +252,37 @@ using the following function:
  .. versionadded:: 3.5
 
 
+For example, a module called ``spam`` would be defined like this::
+
+ static struct PyModuleDef spam_module = {
+ .m_base = PyModuleDef_HEAD_INIT,
+ .m_name = "spam",
+ ...
+ };
+
+ PyMODINIT_FUNC
+ PyInit_spam(void)
+ {
+ return PyModuleDef_Init(&spam_module);
+ }
+
+
 .. _single-phase-initialization:
 
 Legacy single-phase initialization
-..................................
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-.. attention::
- Single-phase initialization is a legacy mechanism to initialize extension
+.. deprecated:: next
+
+ Single-phase initialization is :term:`soft deprecated`.
+ It is a legacy mechanism to initialize extension
  modules, with known drawbacks and design flaws. Extension module authors
  are encouraged to use multi-phase initialization instead.
 
-In single-phase initialization, the
-:ref:`initialization function <extension-export-hook>` (``PyInit_modulename``)
+ However, there are no plans to remove support for it.
+
+In single-phase initialization, the old-style
+:ref:`initializaton function <extension-pyinit>` (``PyInit_modulename``)
 should create, populate and return a module object.
 This is typically done using :c:func:`PyModule_Create` and functions like
 :c:func:`PyModule_AddObjectRef`.
@@ -242,6 +335,8 @@ in the following ways:
 * Single-phase modules support module lookup functions like
  :c:func:`PyState_FindModule`.
 
+* The module's :c:member:`PyModuleDef.m_slots` must be NULL.
+
 .. [#testsinglephase] ``_testsinglephase`` is an internal module used
  in CPython's self-test suite; your installation may or may not
  include it.

Doc/c-api/import.rst

@@ -353,7 +353,7 @@ Importing Modules
  the same as for :c:func:`PyImport_AppendInittab`.
 
  On success, create and return a module object.
- This module will not be initialized; call :c:func:`!PyModule_Exec`
+ This module will not be initialized; call :c:func:`PyModule_Exec`
  to initialize it.
  (Custom importers should do this in their
  :py:meth:`~importlib.abc.Loader.exec_module` method.)

Doc/c-api/init.rst

@@ -1717,7 +1717,8 @@ function. You can create and destroy them using the following functions:
  Only C-level static and global variables are shared between these
  module objects.
 
- * For modules using single-phase initialization,
+ * For modules using legacy
+ :ref:`single-phase initialization <single-phase-initialization>`,
  e.g. :c:func:`PyModule_Create`, the first time a particular extension
  is imported, it is initialized normally, and a (shallow) copy of its
  module's dictionary is squirreled away.