gh-87135: Raise PythonFinalizationError when joining a blocked daemon thread

[encukou]

If Py_IsFinalizing() is true, non-daemon threads (other than the current one) are done, and daemon threads are prevented from acquiring GIL (or thread state), so they cannot finalize themselves and become done. Joining them without timeout would block forever.

Raise PythonFinalizationError instead of hanging.

See gh-123940 for a real-world use case: calling join() from __del__.
Doing this is still ill-advised, but an exception should at least make the issue easier to diagnose.

• Issue: pthread_exit & PyThread_exit_thread from PyEval_RestoreThread etc. are harmful #87135

---

📚 Documentation preview 📚: https://cpython-previews--130402.org.readthedocs.build/

[colesbury]

This seems like a good idea to me.

• What does thread.is_alive() return?
• I think the exception should not be conditional on not having a timeout specified. There's no way it can succeed, so we should just raise the exception immediately like we do when trying to join your own thread. You can also end up with threading.join(timeout=...) calls in an infinite loop.

[vstinner]

I think the exception should not be conditional on not having a timeout specified

I agree with Sam.

[encukou]

What does thread.is_alive() return?

True. Threads that are already done can be joined normally.

I think the exception should not be conditional on not having a timeout specified.

In a finalizer, wouldn't it be OK to wait a bit for graceful termination (using join with a timeout), and then do some teardown regardless of whether the thread survived?
(If Python is being finalized, the thread would of course always survive -- but you might not be writing the code only for that case.)

Raising an exception would mean you skip that teardown, unless you have a try/except around join.

IOW, to me, the reasoning is not as clear-cut here as in the “hang the only Python thread that can run” case.

You can also end up with threading.join(timeout=...) calls in an infinite loop.

Well... You can even write an infinite loop without any join at all! :)
I guess I'm not trying to prevent hangs entirely, just make them easier to diagnose. IMO, a while thread.is_alive() loop is much easier to grok than an internal lock becoming un-acquirable.

---

I wrote the code to hang even with timeout; I'll update the PR if you still think that's the way to go.

[encukou]

Updated to raise even with timeout.

[vstinner]

Is this code path taken by all threads, or only daemon threads?

[encukou]

It's taken by the thread that called Py_FinalizeEx.
When Py_IsFinalizing is true, all other threads than the one that called Py_FinalizeEx are daemonic and they cannot call Python API (including ThreadHandle_join).
So, self must be a daemon thread.

[encukou]

I'll merge on ~Friday if there are no objections.

[encukou]

... And I went offline for a month after writing that.

I'm back now; merging.