Skip to content

gh-112075: Make _PyDict_LoadGlobal thread safe - #117529

Merged
DinoV merged 2 commits into
python:mainfrom
DinoV:nogil_dict_loadglobal
Apr 4, 2024
Merged

gh-112075: Make _PyDict_LoadGlobal thread safe#117529
DinoV merged 2 commits into
python:mainfrom
DinoV:nogil_dict_loadglobal

Conversation

@DinoV

@DinoVDinoV commented Apr 4, 2024

Copy link
Copy Markdown
Contributor

Currently _PyDict_LoadGlobal is using the non-thread safe _Py_dict_lookup and isn't locking the dictionaries. This switches to using the thread safe version and modifies the function to return a new reference.

It also adds an assertion for _Py_dict_lookup that the dictionary should be locked, and fixes up ordered dict to use the thread safe version where the assertion trips.

@DinoV
DinoVforce-pushed the nogil_dict_loadglobal branch from dcd7ede to 548a7c2CompareApril 4, 2024 00:35
@DinoV
DinoV requested a review from colesburyApril 4, 2024 01:03
@DinoV
DinoV marked this pull request as ready for review April 4, 2024 01:03
@DinoVDinoV changed the title Draft gh-112075: Make _PyDict_LoadGlobal thread safegh-112075: Make _PyDict_LoadGlobal thread safeApr 4, 2024
@colesburycolesbury added the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Apr 4, 2024
@bedevere-bot

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @colesbury for commit 548a7c2 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

@bedevere-botbedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Apr 4, 2024

@colesburycolesbury left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment threadObjects/dictobject.c
* key hash failed, key comparison failed, ...). Return NULL if the key doesn't
* exist. Return the value if the key exists.
*
* Returns a new reference.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we will want it to return a possibly deferred reference soon, but this is good for now.

Comment threadObjects/dictobject.c
#ifdef Py_GIL_DISABLED
/* namespace 1: globals */
ix = _Py_dict_lookup(globals, key, hash, &value);
ix = _Py_dict_lookup_threadsafe(globals, key, hash, &value);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this pattern might be simpler if we define _Py_dict_lookup_threadsafe in the default build as _Py_dict_lookup_threadsafe + Py_XNewRef().

We could use it in dict_subscript and dict_get_impl as well.

@DinoV
DinoV merged commit 434bc59 into python:mainApr 4, 2024
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
@DinoV
DinoV deleted the nogil_dict_loadglobal branch May 31, 2024 18:22
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@DinoV@bedevere-bot@colesbury