gh-47146: Soft-deprecate structmember.h, expose its contents via Python.h

[encukou]

The structmember.h header is deprecated, though it continues to be available and there are no plans to remove it. There are no deprecation warnings. Old code can stay unchanged (unless the extra include and non-namespaced macros bother you greatly). Specifically, no uses in CPython are updated -- that would just be unnecessary churn.

The header's contents are now available just by including Python.h, with a PY_ prefix added if it was missing:

• PyMemberDef, PyMember_GetOne, PyMember_SetOne
• Type macros like PY_T_INT, PY_T_DOUBLE, etc. (previously T_INT, T_DOUBLE, etc.)
• The flags PY_READONLY (previously READONLY) and PY_AUDIT_READ (name unchanged)

Several items are not exposed from Python.h:

• T_OBJECT (use PY_T_OBJECT_EX)
• T_NONE (previously undocumented, and pretty quirky)
• The macro WRITE_RESTRICTED which does nothing.
• The macros RESTRICTED and READ_RESTRICTED, equivalents of PY_AUDIT_READ.
• In some configurations, <stddef.h> is not included from Python.h.
  It should be included manually when using offsetof().

The PY_T_*, PY_READONLY and PY_AUDIT_READ macros are added to the stable API manifest. This is just a clerical change, really -- Stable ABI extensions in the wild use structmember.h, and PyMemberDef & Py_tp_members are already listed.

There is discussion on the issue to rename T_PYSSIZET to PY_T_SSIZE or similar. I chose not to do that -- users will probably copy/paste that with any spelling, and not renaming it makes migration docs simpler.

Co-Authored-By: Alexander Belopolsky abalkin@users.noreply.github.com
Co-Authored-By: Matthias Braun MatzeB@users.noreply.github.com

• Issue: PyMemberDef missing in limited API / Deprecate structmember.h #47146

[encukou]

Today I noticed the style guide says macros should have the Py_ prefix, not PY_. A minor point, but since this is renaming a bunch of macros, I'll switch to the preferred form.

[encukou]

The PY_T_*, PY_READONLY and PY_AUDIT_READ macros are added to the stable API manifest. This is just a clerical change, really -- Stable ABI extensions in the wild use structmember.h, and PyMemberDef & Py_tp_members are already listed.

And by the same token, PyMember_GetOne and PyMember_SetOne should be added too.

[encukou]

And here are tests.
I'd appreciate a review, if anyone can spare the time.

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @encukou for commit e44b7ca 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

[erlend-aasland]

Looks good. I did some nitpicking.

[encukou]

Thank you! I did the changes, and will merge once everything is green.

[vstinner]

Nice work!

[serhiy-storchaka]

Isn't it strange that the only public constant for object has an _EX suffix? Why not rename Py_T_OBJECT_EX to Py_T_OBJECT?

[vstinner]

I agree that _Py_T_OBJECT is surprising: returning None if an attribute doesn't exist, so it's not possible to know if the attribute value is None or if the attribute doesn't exist.

Py_T_OBJECT_EX behavior of raising an AttributeError if the attribute value is NULL doesn't better. I agree with @serhiy-storchaka, it should just be Py_T_OBJECT.

By the way, how was _Py_T_NONE used in the past? It's documented as:

Deprecated. Value is always None.

[serhiy-storchaka]

Both _Py_T_OBJECT and Py_T_OBJECT_EX correspond to behavior of attributes of instances of Python classes. Just the latter is the default behavior, and the former is for the case when an instance attribute shadows the class attribute set to None. See an example in #107253.

T_NONE was introduced in 50e9fb9. I have not found a single case of its use.

[vstinner]

T_NONE was introduced in 50e9fb9. I have not found a single case of its use.

Maybe it's not worth it to add _Py_NONE in this case. I don't know.

[encukou]

Isn't it strange that the only public constant for object has an _EX suffix? Why not rename Py_T_OBJECT_EX to Py_T_OBJECT?

T_OBJECT is still public API, it's just discouraged. (Probably not properly “soft-deprecated” in the new sense though?)
Since all the others add Py_, I thought it would be confusing to also drop the suffix in this case.