Skip to content

gh-104341: Call _PyEval_ReleaseLock() with NULL When Finalizing the Current Thread - #105109

Merged
ericsnowcurrently merged 7 commits into
python:mainfrom
ericsnowcurrently:fix-threading-module-shutdown-3.12
Jun 1, 2023
Merged

gh-104341: Call _PyEval_ReleaseLock() with NULL When Finalizing the Current Thread#105109
ericsnowcurrently merged 7 commits into
python:mainfrom
ericsnowcurrently:fix-threading-module-shutdown-3.12

Conversation

@ericsnowcurrently

@ericsnowcurrentlyericsnowcurrently commented May 30, 2023•
edited by bedevere-bot
Loading

Copy link
Copy Markdown
Member

This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads.

This is a much simpler approach to solving the race than in other PRs I've posted. I'd still like to pursue some of those other ideas but that can be done separately for 3.13+.

(The idea for this approach came out of discussions with @markshannon.)

@bedevere-botbedevere-bot mentioned this pull request May 30, 2023
Closed
ericsnowcurrently
ericsnowcurrently commented May 30, 2023
View reviewed changes
Comment threadPython/ceval_gil.c
Comment threadPython/ceval_gil.c

void
_PyEval_ReleaseLock(PyThreadState *tstate)
_PyEval_ReleaseLock(PyInterpreterState *interp, PyThreadState *tstate)

@ericsnowcurrentlyericsnowcurrentlyMay 30, 2023

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other key part of this change is here, where we pass in the interpreter separately from the thread state, which allows tstate to be NULL.

@ericsnowcurrently
ericsnowcurrently marked this pull request as ready for review May 30, 2023 21:10
gpshead
gpshead reviewed May 30, 2023
View reviewed changes
Comment threadPython/ceval_gil.c
Comment threadPython/pystate.c Outdated
@ericsnowcurrentlyericsnowcurrently mentioned this pull request May 31, 2023
Merged
pitrou
pitrou reviewed May 31, 2023
View reviewed changes
Comment threadPython/ceval_gil.c Outdated
static void
drop_gil(struct _ceval_state *ceval, PyThreadState *tstate)
{
/* We shouldn't be using a thread state that isn't viable any more. */

@pitroupitrouMay 31, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is cryptic here. It doesn't say why it's here nor in which situation the "non-viable thread state" occurs.

@ericsnowcurrentlyericsnowcurrentlyMay 31, 2023

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've clarified the comment.

@ericsnowcurrently

ericsnowcurrently commented Jun 1, 2023

Copy link
Copy Markdown
MemberAuthor

@gpshead, @pitrou, do you have any objections to this PR at this point? I'd like to land this ASAP for beta 2. Thanks!

@gpshead

gpshead commented Jun 1, 2023

Copy link
Copy Markdown
Member

any objections to this PR at this point? I'd like to land this ASAP for beta 2. Thanks!

I think we're good to go: no objections from me. I never assume I can have full confidence about modifications to this code, but CI and buildbots and ultimately beta2 testing should reveal more if anything else lurks...

@ericsnowcurrently

ericsnowcurrently commented Jun 1, 2023

Copy link
Copy Markdown
MemberAuthor

Thanks you!

@ericsnowcurrently
ericsnowcurrently merged commit 3698fda into python:mainJun 1, 2023
@miss-islington

miss-islington commented Jun 1, 2023

Copy link
Copy Markdown
Contributor

Thanks @ericsnowcurrently for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

@miss-islington

miss-islington commented Jun 1, 2023

Copy link
Copy Markdown
Contributor

Sorry @ericsnowcurrently, I had trouble checking out the 3.12 backport branch.
Please retry by removing and re-adding the "needs backport to 3.12" label.
Alternatively, you can backport using cherry_picker on the command line.
cherry_picker 3698fda06eefb3c01e78c4c07f46fcdd0559e0f6 3.12

@ericsnowcurrently
ericsnowcurrently deleted the fix-threading-module-shutdown-3.12 branch June 1, 2023 22:24
@ericsnowcurrentlyericsnowcurrently added needs backport to 3.12 only security fixes and removed needs backport to 3.12 only security fixes labels Jun 1, 2023
@miss-islington

miss-islington commented Jun 1, 2023

Copy link
Copy Markdown
Contributor

Thanks @ericsnowcurrently for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 1, 2023
@ericsnowcurrently@miss-islington
pythongh-104341: Call _PyEval_ReleaseLock() with NULL When Finalizing…
d7f46a8
… the Current Thread (pythongh-105109)
This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads.
(The idea for this approach came out of discussions with @markshannon.)
(cherry picked from commit 3698fda)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
@bedevere-bot

bedevere-bot commented Jun 1, 2023

Copy link
Copy Markdown

GH-105209 is a backport of this pull request to the 3.12 branch.

@bedevere-botbedevere-bot removed the needs backport to 3.12 only security fixes label Jun 1, 2023
ericsnowcurrently pushed a commit that referenced this pull request Jun 1, 2023
@miss-islington
[3.12]gh-104341: Call _PyEval_ReleaseLock() with NULL When Finalizin…
d2be5c7
…g the Current Thread (gh-105109) (gh-105209)
This avoids the problematic race in drop_gil() by skipping the FORCE_SWITCHING code there for finalizing threads.
(The idea for this approach came out of discussions with @markshannon.)
(cherry picked from commit 3698fda)
Co-authored-by: Eric Snow ericsnowcurrently@gmail.com
@erlend-aaslanderlend-aasland mentioned this pull request Jun 6, 2023
Closed
@ericsnowcurrentlyericsnowcurrently mentioned this pull request Oct 25, 2023
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pitroupitroupitrou left review comments
@gpsheadgpsheadgpshead approved these changes
@markshannonmarkshannonAwaiting requested review from markshannon
@tim-onetim-oneAwaiting requested review from tim-one
@Yhg1sYhg1sAwaiting requested review from Yhg1s

Assignees

@ericsnowcurrentlyericsnowcurrently

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@ericsnowcurrently@gpshead@miss-islington@bedevere-bot@pitrou