Skip to content

gh-140431: Fix GC crash due to partially initialized coroutines - #140470

Merged
colesbury merged 1 commit into
python:mainfrom
colesbury:gh-140431-gen
Oct 23, 2025
Merged

gh-140431: Fix GC crash due to partially initialized coroutines#140470
colesbury merged 1 commit into
python:mainfrom
colesbury:gh-140431-gen

Conversation

@colesbury

@colesburycolesbury commented Oct 22, 2025

Copy link
Copy Markdown
Contributor

The make_gen() function creates and tracks generator/coro objects, but doesn't fully initialize all the fields. At a minimum, we need to initialize all the fields that may be accessed by gen_traverse because the call to compute_cr_origin() can trigger a GC.

The `make_gen()` function creates and tracks generator/coro objects, but
doesn't fully initialize all the fields. At a minimum, we need to
initialize all the fields that may be accessed by gen_traverse because
the call to `compute_cr_origin()` can trigger a GC.
@pgdr

pgdr commented Oct 23, 2025

Copy link
Copy Markdown
Contributor

For what it's worth, with this patch, I am no longer able to reproduce the bug.

With this patch, I was not able to reproduce the bug after an hour of running the tests.

Without this patch, the segfault occurs on average after < 4 minutes.

I have used the following script to check:

Details
#!/usr/bin/env bashset -e
time_start=$(date +%s)
git clean -dfx
CC=clang CXX=clang++ ./configure --with-address-sanitizer --with-pydebug --disable-gil
make -j12
./python -X dev -m test -R 9:9 test_asyncio.test_free_threading
./python -X dev -m test -R 9:9 test_asyncio.test_free_threading
./python -X dev -m test -R 9:9 test_asyncio.test_free_threading
./python -X dev -m test -R 9:9 test_asyncio.test_free_threading
./python -X dev -m test -R 9:9 test_asyncio.test_free_threading
./python -X dev -m test -R 9:9 test_asyncio.test_free_threading
./python -X dev -m test -R 9:9 test_asyncio.test_free_threading
./python -X dev -m test -R 9:9 test_asyncio.test_free_threading
./python -X dev -m test -R 9:9 test_asyncio.test_free_threading
./python -X dev -m test -R 9:9 test_asyncio.test_free_threading
time_end=$(date +%s)
delta=$((time_end - time_start))
minutes=$((delta /60))
seconds=$((delta %60))echoechoecho"Elapsed time: ${minutes}m ${seconds}s"echoecho -n "Possibly good commit: "
git rev-parse HEAD
git log --oneline | head -n 1
echo

@efimov-mikhailefimov-mikhail left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

We definitely need to set correct value of gen->gi_iframe.f_executable in make_gen.

@kumaraditya303kumaraditya303 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@colesbury
colesbury merged commit 574405c into python:mainOct 23, 2025
53 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @colesbury for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

@colesbury
colesbury deleted the gh-140431-gen branch October 23, 2025 14:18
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 23, 2025
…pythongh-140470)
The `make_gen()` function creates and tracks generator/coro objects, but
doesn't fully initialize all the fields. At a minimum, we need to
initialize all the fields that may be accessed by gen_traverse because
the call to `compute_cr_origin()` can trigger a GC.
(cherry picked from commit 574405c)
Co-authored-by: Sam Gross <colesbury@gmail.com>
@bedevere-app

Copy link
Copy Markdown

GH-140504 is a backport of this pull request to the 3.14 branch.

@bedevere-appbedevere-appBot removed the needs backport to 3.14 bugs and security fixes label Oct 23, 2025
@bedevere-bot

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot ARM64 macOS 3.x (tier-2) has failed when building commit 574405c.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/725/builds/12170) and take a look at the build logs.
  4. Check if the failure is related to this commit (574405c) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/725/builds/12170

Failed tests:

  • test_urllib2net

Failed subtests:

  • test_ftp_timeout - test.test_urllib2net.TimeoutTest.test_ftp_timeout

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 20, in _retry_thricereturn func(*args, **kwargs)
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 187, in urlopenreturn opener.open(url, data, timeout)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 487, in open
response =self._open(req, data)
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 504, in _open
result =self._call_chain(self.handle_open, protocol, protocol +'_open', req)
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 464, in _call_chain
result = func(*args)
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1556, in ftp_openraise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: [Errno 60] Operation timed out>
Traceback (most recent call last):
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 22, in _retry_thrice
last_exc = e
^^^^^^^^ResourceWarning: unclosed <socket.socket fd=7, family=2, type=1, proto=6, laddr=('192.168.0.101', 54826), raddr=('68.183.26.59', 21)>
Warning -- Unraisable exception
Exception ignored while finalizing socket <socket.socket fd=8, family=2, type=1, proto=6, laddr=('192.168.0.101', 54828), raddr=('68.183.26.59', 21)>:
Traceback (most recent call last):
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1546, in ftp_open
fp, retrlen = fw.retrfile(file, type)
~~~~~~~~~~~^^^^^^^^^^^^
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1826, in retrfile
conn, retrlen =self.ftp.ntransfercmd(cmd)
~~~~~~~~~~~~~~~~~~~~~^^^^^
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/ftplib.py", line 354, in ntransfercmd
conn = socket.create_connection((host, port), self.timeout,
source_address=self.source_address)
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 879, in create_connectionraise exceptions[0]
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 864, in create_connection
sock.connect(sa)
~~~~~~~~~~~~^^^^TimeoutError: [Errno 60] Operation timed out
Traceback (most recent call last):
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/support/__init__.py", line 847, in gc_collect
gc.collect()
~~~~~~~~~~^^ResourceWarning: unclosed <socket.socket fd=6, family=2, type=1, proto=6, laddr=('192.168.0.101', 54818), raddr=('68.183.26.59', 21)>
Traceback (most recent call last):
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 22, in _retry_thrice
last_exc = e
^^^^^^^^ResourceWarning: unclosed <socket.socket fd=8, family=2, type=1, proto=6, laddr=('192.168.0.101', 54828), raddr=('68.183.26.59', 21)>
ERROR
Traceback (most recent call last):
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/libregrtest/single.py", line 165, in _load_run_test
regrtest_runner(result, test_func, runtests)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/libregrtest/single.py", line 118, in regrtest_runner
test_result = test_func()
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/libregrtest/single.py", line 162, in test_funcreturn run_unittest(test_mod, runtests)
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/libregrtest/single.py", line 42, in run_unittestreturn _run_suite(tests)
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/libregrtest/single.py", line 105, in _run_suiteraise support.TestFailedWithDetails(err, errors, failures, stats=stats)
Traceback (most recent call last):
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 22, in _retry_thrice
last_exc = e
^^^^^^^^ResourceWarning: unclosed <socket.socket fd=6, family=2, type=1, proto=6, laddr=('192.168.0.101', 54274), raddr=('68.183.26.59', 21)>
Warning -- Unraisable exception
Exception ignored while finalizing socket <socket.socket fd=8, family=2, type=1, proto=6, laddr=('192.168.0.101', 54621), raddr=('68.183.26.59', 21)>:
Traceback (most recent call last):
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1546, in ftp_open
fp, retrlen = fw.retrfile(file, type)
~~~~~~~~~~~^^^^^^^^^^^^
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1826, in retrfile
conn, retrlen =self.ftp.ntransfercmd(cmd)
~~~~~~~~~~~~~~~~~~~~~^^^^^
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/ftplib.py", line 354, in ntransfercmd
conn = socket.create_connection((host, port), self.timeout,
source_address=self.source_address)
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 879, in create_connectionraise exceptions[0]
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 864, in create_connection
sock.connect(sa)
~~~~~~~~~~~~^^^^TimeoutError: [Errno 60] Operation timed out
Traceback (most recent call last):
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 22, in _retry_thrice
last_exc = e
^^^^^^^^ResourceWarning: unclosed <socket.socket fd=8, family=2, type=1, proto=6, laddr=('192.168.0.101', 54621), raddr=('68.183.26.59', 21)>
ERROR
Traceback (most recent call last):
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/support/__init__.py", line 847, in gc_collect
gc.collect()
~~~~~~~~~~^^ResourceWarning: unclosed <socket.socket fd=7, family=2, type=1, proto=6, laddr=('192.168.0.101', 54830), raddr=('68.183.26.59', 21)>
Traceback (most recent call last):
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1546, in ftp_open
fp, retrlen = fw.retrfile(file, type)
~~~~~~~~~~~^^^^^^^^^^^^
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1826, in retrfile
conn, retrlen =self.ftp.ntransfercmd(cmd)
~~~~~~~~~~~~~~~~~~~~~^^^^^
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/ftplib.py", line 354, in ntransfercmd
conn = socket.create_connection((host, port), self.timeout,
source_address=self.source_address)
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 879, in create_connectionraise exceptions[0]
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 864, in create_connection
sock.connect(sa)
~~~~~~~~~~~~^^^^TimeoutError: [Errno 60] Operation timed out
Traceback (most recent call last):
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 351, in test_ftp_timeout
u = _urlopen_with_retry(self.FTP_HOST, timeout=60)
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 28, in wrappedreturn _retry_thrice(func, exc, *args, **kwargs)
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 24, in _retry_thriceraise last_exc
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 20, in _retry_thricereturn func(*args, **kwargs)
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 187, in urlopenreturn opener.open(url, data, timeout)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 487, in open
response =self._open(req, data)
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 504, in _open
result =self._call_chain(self.handle_open, protocol, protocol +'_open', req)
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 464, in _call_chain
result = func(*args)
File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1556, in ftp_openraise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: [Errno 60] Operation timed out>

colesbury added a commit that referenced this pull request Oct 23, 2025
gh-140470) (gh-140504)
The `make_gen()` function creates and tracks generator/coro objects, but
doesn't fully initialize all the fields. At a minimum, we need to
initialize all the fields that may be accessed by gen_traverse because
the call to `compute_cr_origin()` can trigger a GC.
(cherry picked from commit 574405c)
Co-authored-by: Sam Gross <colesbury@gmail.com>
StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull request Dec 6, 2025
…pythongh-140470)
The `make_gen()` function creates and tracks generator/coro objects, but
doesn't fully initialize all the fields. At a minimum, we need to
initialize all the fields that may be accessed by gen_traverse because
the call to `compute_cr_origin()` can trigger a GC.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@colesbury@pgdr@bedevere-bot@efimov-mikhail@kumaraditya303