Skip to content

Add guidelines for changing/removing from the Limited API#778

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
encukou merged 4 commits into from
Dec 14, 2021
Merged

Add guidelines for changing/removing from the Limited API #778

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
58bf107
Add guidelines for changing/removing from the Limited API
encukou Dec 7, 2021
bae3bb1
Link to a tag rather than a branch
encukou Dec 7, 2021
b7f731f
Add more backticks
encukou Dec 14, 2021
e751783
Undo unrelated change
encukou Dec 14, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 38 additions & 2 deletions c-api.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -101,8 +101,44 @@ No changes that break the Stable ABI are allowed.
The Limited API should be defined in ``Include/``, excluding the
``cpython`` and ``internal`` subdirectories.

Guidelines for changing the Limited API
---------------------------------------

Guidelines for changing the Limited API, and removing items from it
-------------------------------------------------------------------

While the *Stable ABI* must not be broken, the existing Limited API can be
changed, and items can be removed from it, if:

- the Backwards Compatibility Policy (:pep:`387`) is followed, and
- the Stable ABI is not broken -- that is, extensions compiled with
Limited API of older versions of Python continue to work on
newer versions of Python.

@vstinnervstinnerDec 9, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to export a function which always fail, if the function has been removed? Removing PyUnicode_InternImmortal() sounds like an interesting challenge: https://bugs.python.org/issue41692

Maybe the function can be replaced with a function... which does nothing but log an error?

@arhadthedevarhadthedevDec 9, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to export a function which always fail, if the function has been removed?

I believe, only if the function is not vital for already compiled modules (so its failure won't hamper their main activity). That's actually the whole point of backwards compatibility, to not break third party things that already work.

@encukouencukouDec 9, 2021

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, after it's removed from the API and when SSTATE_INTERNED_IMMORTAL needs to be removed, PyUnicode_InternImmortal can do an unpaired INCREF: make the string immortal, but leak it.
See https://www.python.org/dev/peps/pep-0652/#stable-abi:

Future Python versions may deprecate some members of the Stable ABI. Deprecated members will still work, but may suffer from issues like reduced performance or, in the most extreme cases, memory/resource leaks.


This is tricky to do and requires careful thought.
Some examples:

- Functions, structs etc. accessed by macros in *any version* of the
Limited API are part of the Stable ABI, even if they are named with
an underscore. They must not be removed and their signature must not change.
(Their implementation may change, though.)
- Structs members cannot be rearranged if they were part of any version of
the Limited API.
- If the Limited API allows users to allocate a struct directly,
its size must not change.
- Exported symbols (functions and data) must continue to be available
as exported symbols. Specifically, a function can only be converted
to a ``static inline`` function (or macro) if Python also continues to
provide the actual function.
For an example, see the ``Py_NewRef`` `macro`_ and `redefinition`_ in 3.10.

.. _macro: https://github.com/python/cpython/blob/v3.10.1/Include/object.h#L591-L595
.. _redefinition: https://github.com/python/cpython/blob/v3.10.1/Objects/object.c#L2298-L2308

It is possible to remove items marked as part of the Stable ABI, but only
if there was no way to use them in any past version of the Limited API.


Guidelines for adding to the Limited API
----------------------------------------

- Guidelines for the general :ref:`public-capi` apply.

Expand Down