gh-109746: Make _thread.start_new_thread delete state of new thread on its startup failure

[chgnrdv]

Fixes#109746.

If Python fails to start newly created thread due to failure of underlying PyThread_start_new_thread() call, its state should be removed from interpreter' thread states list to avoid its double cleanup.

• make PyThreadState_Delete accept unbound thread state
• in thread_PyThread_start_new_thread, delete state of new thread from interpreter if PyThread_start_new_thread call fails

• Issue: Crash at finalization after fail to start new thread #109746

[serhiy-storchaka]

The test did not fail on non-patched code, failed on Windows, and hanged on macOS. I rewrote it. According to a comment in a similar test, setrlimit() does not work on macOS. Will see if it works for this test.

[serhiy-storchaka]

!buildbot bsd

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @serhiy-storchaka for commit 1f3ecd9 🤖

The command will test the builders whose names match following regular expression: bsd

The builders matched are:

• AMD64 FreeBSD14 PR
• AMD64 FreeBSD15 PR
• AMD64 FreeBSD PR

[devdanzin]

Applying a version of the fix in this PR solves an issue I had where all processes with large thread counts would abort at finalization (particularly annoying when you're trying to fuzz CPython).

[serhiy-storchaka]

!buildbot bsd

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @serhiy-storchaka for commit 185d5a6 🤖

The command will test the builders whose names match following regular expression: bsd

The builders matched are:

• AMD64 FreeBSD Refleaks PR
• AMD64 FreeBSD14 PR
• AMD64 FreeBSD15 PR
• AMD64 FreeBSD PR

[serhiy-storchaka]

!buildbot bsd

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @serhiy-storchaka for commit 55a8237 🤖

The command will test the builders whose names match following regular expression: bsd

The builders matched are:

• AMD64 FreeBSD Refleaks PR
• AMD64 FreeBSD14 PR
• AMD64 FreeBSD15 PR
• AMD64 FreeBSD PR

[miss-islington-app]

Thanks @chgnrdv for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

[miss-islington-app]

Sorry, @chgnrdv and @serhiy-storchaka, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker ca3ea9ad05c3d876a58463595e5b4228fda06936 3.13

[bedevere-app]

GH-127171 is a backport of this pull request to the 3.13 branch.

[bedevere-app]

GH-127173 is a backport of this pull request to the 3.12 branch.

[vstinner]

!buildbot bsd

The test is unstable on macOS and FreeBSD: it fails randomly. Examples:

• macOS: https://buildbot.python.org/#/builders/1449/builds/713
• FreeBSD: https://buildbot.python.org/#/builders/1223/builds/4680

Example of failure on FreeBSD:

FAIL: test_start_new_thread_failed (test.test_threading.ThreadTests.test_start_new_thread_failed)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/buildbot/buildarea/3.x.ware-freebsd/build/Lib/test/test_threading.py", line 1205, in test_start_new_thread_failed
self.assertEqual(out, b'ok')
~~~~~~~~~~~~~~~~^^^^^^^^^^^^
AssertionError: b"skipshouldn't be printed" != b'ok'

[vstinner]

On Linux, RLIMIT_NPROC is the maximum number of threads. On FreeBSD, RLIMIT_NPROC is the maximum number of processes.

[vstinner]

I suggest to make the test specific to Linux, or use another approach than RLIMIT_NPROC.

[serhiy-storchaka]

Ah, there is a race condition here. Should be fixed by #127299.