Skip to content

gh-127545: Replace _Py_ALIGN_AS(V) by _Py_ALIGNED_DEF(N, T) - #135209

Merged
encukou merged 6 commits into
python:mainfrom
encukou:gc_crash
Jun 11, 2025
Merged

gh-127545: Replace _Py_ALIGN_AS(V) by _Py_ALIGNED_DEF(N, T)#135209
encukou merged 6 commits into
python:mainfrom
encukou:gc_crash

Conversation

@encukou

@encukouencukou commented Jun 6, 2025•
edited
Loading

Copy link
Copy Markdown
Member

_Py_ALIGNED_DEF(N, T) might be a better way to make the various _Alignas alternatives,
which behave in interesting ways, increase (rather than set) a variable's alignment.

The standard alignas/_Alignas error out if they would decrease the alignment. To prevent that, they can be used again with the defined variable's type.

The standard alignas/_Alignas can't be used on a struct definition, the workaround is to use it on one of the struct's members.

MSVC declspec(aligned) only takes integer literals.

MSVC declspec(aligned) had a bug when applied to a combined struct+member definition; a workaround is to separate the struct definition. This avoids a potentially ABI-breaking bug in older MSVC versions (which newer versions warn about, using code C5274).
This PR does that for PyASCIIObject.state.

fthainand others added 2 commits June 2, 2025 10:37
@fthain
Specify minimum PyGC_Head and PyObject alignment to fix build failure
392f6c5
As documented in InternalDocs/garbage_collector.md, the garbage collector
stores flags in the least significant two bits of the _gc_prev pointer
in struct PyGC_Head. Consequently, this pointer is only capable of storing
a location that's aligned to a 4-byte boundary.
This alignment requirement is documented but it's not actually encoded.
The code only works when python happens to run on a platform that has a
sufficiently large minimum alignment for the structs in question.
The same problem arises with PyObject pointers because the least
significant bits get used for PyStackRef tags.
Since we know that 2 bits are needed, we also know the minimum alignment
that's needed. Let's make that explicit, so the compiler can then make
those bits available.
This patch fixes a segfault in _bootstrap_python. In 3.14.0 beta 2
this fixes the "Assertion `!PyStackRef_IsTaggedInt(ref)' failed" when
built with --config-pydebug.
Also, making the requirements explicit improves clarity.
This bug was previously investigated by Adrian Glaubitz here:
https://lists.debian.org/debian-68k/2024/11/msg00020.htmlhttps://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1087600
Although Adrian's patch isn't really correct (because natural alignment
is not needed), he deserves full credit for finding the root cause.
@encukou
Replace _Py_ALIGN_AS(V) by _Py_ALIGNED_DEF(N, T)
e0156b0
This is a common façade for the various `_Alignas` alternatives,
which behave in interesting ways.
The standard `alignas` errors if it would decrease the alignment;
to prevent that it can be used again with the defined variable's
type.
The standard `alignas` can't be used on a `struct` definition,
the workaround is to use it on one of the `struct`'s members.
MSVC `declspec(aligned)` only takes integer literals.
MSVC `declspec(aligned)` had a bug when applied to a combined
struct+member definition.
The workaround is to separate the struct definition.
Do that for `PyASCIIObject.state`.
@bedevere-appbedevere-appBot mentioned this pull request Jun 6, 2025
Closed
@encukouencukou mentioned this pull request Jun 6, 2025
Closed
@encukouencukou added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jun 6, 2025
@bedevere-bot

bedevere-bot commented Jun 6, 2025

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @encukou for commit 512fe58 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F135209%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-botbedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jun 6, 2025
@encukou

encukou commented Jun 7, 2025•
edited
Loading

Copy link
Copy Markdown
MemberAuthor

@fthain, would you be able to test this on a m68k?

[edit] Buildbot failures look unrelated.

This was referenced Jun 7, 2025
Closed
Closed
encukou
encukou commented Jun 7, 2025
View reviewed changes
Comment threadInclude/pymacro.h Outdated
encukou
encukou commented Jun 7, 2025
View reviewed changes
Comment threadInclude/pymacro.h Outdated
@fthain

fthain commented Jun 10, 2025

Copy link
Copy Markdown
Contributor

@fthain, would you be able to test this on a m68k?

I tested your branch using Gentoo/m68k. It works fine. I still see some test suite failures but these are not regressions. Thanks for your help with this bug, @encukou.

@encukou
encukou marked this pull request as ready for review June 10, 2025 06:39
vstinner
vstinner approved these changes Jun 10, 2025
View reviewed changes

@vstinnervstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Just try to clarify that the number is a number of bytes, and not a number of bits.

Comment threadInclude/pymacro.h Outdated
Comment threadInclude/object.h Outdated
@encukou@vstinner
Clarify units
6d7c805
Co-authored-by: Victor Stinner <vstinner@python.org>
@ZeroIntensityZeroIntensity mentioned this pull request Jun 10, 2025
Open
@encukou
encukou merged commit 49d7236 into python:mainJun 11, 2025
@encukou
encukou deleted the gc_crash branch June 11, 2025 10:45
lkollar pushed a commit to lkollar/cpython that referenced this pull request Jun 19, 2025
@encukou@fthain
@vstinner@glaubitz@lkollar
pythongh-127545: Add _Py_ALIGNED_DEF(N, T) and use it for PyObject (p…
460fcf1
…ythonGH-135209)
* Replace _Py_ALIGN_AS(V) by _Py_ALIGNED_DEF(N, T)
This is now a common façade for the various `_Alignas` alternatives,
which behave in interesting ways -- see the source comment.
The new macro (and MSVC's `__declspec(align)`) should not be used
on a variable/member declaration that includes a struct declaraton.
A workaround is to separate the struct definition.
Do that for `PyASCIIObject.state`.
* Specify minimum PyGC_Head and PyObject alignment
As documented in InternalDocs/garbage_collector.md, the garbage collector
stores flags in the least significant two bits of the _gc_prev pointer
in struct PyGC_Head. Consequently, this pointer is only capable of storing
a location that's aligned to a 4-byte boundary.
Encode this requirement using _Py_ALIGNED_DEF.
This patch fixes a segfault in m68k, which was previously investigated
by Adrian Glaubitz here:
https://lists.debian.org/debian-68k/2024/11/msg00020.htmlhttps://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1087600
Original patch (using the GCC-only Py_ALIGNED) by Finn Thain.
Co-authored-by: Finn Thain <fthain@linux-m68k.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
Pranjal095 pushed a commit to Pranjal095/cpython that referenced this pull request Jul 12, 2025
@encukou@fthain
@vstinner@glaubitz@Pranjal095
pythongh-127545: Add _Py_ALIGNED_DEF(N, T) and use it for PyObject (p…
ed93144
…ythonGH-135209)
* Replace _Py_ALIGN_AS(V) by _Py_ALIGNED_DEF(N, T)
This is now a common façade for the various `_Alignas` alternatives,
which behave in interesting ways -- see the source comment.
The new macro (and MSVC's `__declspec(align)`) should not be used
on a variable/member declaration that includes a struct declaraton.
A workaround is to separate the struct definition.
Do that for `PyASCIIObject.state`.
* Specify minimum PyGC_Head and PyObject alignment
As documented in InternalDocs/garbage_collector.md, the garbage collector
stores flags in the least significant two bits of the _gc_prev pointer
in struct PyGC_Head. Consequently, this pointer is only capable of storing
a location that's aligned to a 4-byte boundary.
Encode this requirement using _Py_ALIGNED_DEF.
This patch fixes a segfault in m68k, which was previously investigated
by Adrian Glaubitz here:
https://lists.debian.org/debian-68k/2024/11/msg00020.htmlhttps://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1087600
Original patch (using the GCC-only Py_ALIGNED) by Finn Thain.
Co-authored-by: Finn Thain <fthain@linux-m68k.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
@encukouencukou mentioned this pull request Aug 1, 2025
Closed
taegyunkim pushed a commit to taegyunkim/cpython that referenced this pull request Aug 4, 2025
@encukou@fthain
@vstinner@glaubitz@taegyunkim
pythongh-127545: Add _Py_ALIGNED_DEF(N, T) and use it for PyObject (p…
fca08c8
…ythonGH-135209)
* Replace _Py_ALIGN_AS(V) by _Py_ALIGNED_DEF(N, T)
This is now a common façade for the various `_Alignas` alternatives,
which behave in interesting ways -- see the source comment.
The new macro (and MSVC's `__declspec(align)`) should not be used
on a variable/member declaration that includes a struct declaraton.
A workaround is to separate the struct definition.
Do that for `PyASCIIObject.state`.
* Specify minimum PyGC_Head and PyObject alignment
As documented in InternalDocs/garbage_collector.md, the garbage collector
stores flags in the least significant two bits of the _gc_prev pointer
in struct PyGC_Head. Consequently, this pointer is only capable of storing
a location that's aligned to a 4-byte boundary.
Encode this requirement using _Py_ALIGNED_DEF.
This patch fixes a segfault in m68k, which was previously investigated
by Adrian Glaubitz here:
https://lists.debian.org/debian-68k/2024/11/msg00020.htmlhttps://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1087600
Original patch (using the GCC-only Py_ALIGNED) by Finn Thain.
Co-authored-by: Finn Thain <fthain@linux-m68k.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
Agent-Hellboy pushed a commit to Agent-Hellboy/cpython that referenced this pull request Aug 19, 2025
@encukou@fthain
@vstinner@glaubitz@Agent-Hellboy
pythongh-127545: Add _Py_ALIGNED_DEF(N, T) and use it for PyObject (p…
a80d586
…ythonGH-135209)
* Replace _Py_ALIGN_AS(V) by _Py_ALIGNED_DEF(N, T)
This is now a common façade for the various `_Alignas` alternatives,
which behave in interesting ways -- see the source comment.
The new macro (and MSVC's `__declspec(align)`) should not be used
on a variable/member declaration that includes a struct declaraton.
A workaround is to separate the struct definition.
Do that for `PyASCIIObject.state`.
* Specify minimum PyGC_Head and PyObject alignment
As documented in InternalDocs/garbage_collector.md, the garbage collector
stores flags in the least significant two bits of the _gc_prev pointer
in struct PyGC_Head. Consequently, this pointer is only capable of storing
a location that's aligned to a 4-byte boundary.
Encode this requirement using _Py_ALIGNED_DEF.
This patch fixes a segfault in m68k, which was previously investigated
by Adrian Glaubitz here:
https://lists.debian.org/debian-68k/2024/11/msg00020.htmlhttps://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1087600
Original patch (using the GCC-only Py_ALIGNED) by Finn Thain.
Co-authored-by: Finn Thain <fthain@linux-m68k.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinnervstinnervstinner approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@encukou@bedevere-bot@fthain@vstinner