Skip to content

gh-128002: fix many thread safety issues in asyncio - #128147

Merged
kumaraditya303 merged 26 commits into
python:mainfrom
kumaraditya303:asyncio-tsafe
Jan 4, 2025
Merged

gh-128002: fix many thread safety issues in asyncio#128147
kumaraditya303 merged 26 commits into
python:mainfrom
kumaraditya303:asyncio-tsafe

Conversation

@kumaraditya303

@kumaraditya303kumaraditya303 commented Dec 21, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Changes:

  • Makes _asyncio.Task and _asyncio.Future thread-safe by adding critical sections
  • Add assertions to check for thread safety checking locking of object by critical sections in internal functions
  • Make _asyncio.all_tasks thread safe when eager tasks are used
  • Change asyncio state lock to a mutex rather than critical section to fix re-entrancy crash, not really sure about this one
  • Add a thread safety test

@bedevere-appbedevere-appBot mentioned this pull request Dec 21, 2024
Closed
9 tasks
@kumaraditya303

kumaraditya303 commented Dec 31, 2024•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

@colesbury While running the tests locally on forever mode I see frequent tsan warnings and rarely some crashes in exceptions.c arising from things like setting traceback on a exception, should I add those to the suppressions list?

colesbury
colesbury reviewed Jan 2, 2025
View reviewed changes

@colesburycolesbury left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still reading through the other locking changes, but a comment on all_tasks below:

Comment threadModules/_asynciomodule.c Outdated
Comment threadModules/_asynciomodule.c Outdated
@kumaraditya303kumaraditya303 mentioned this pull request Jan 2, 2025
Closed
@kumaraditya303

kumaraditya303 commented Jan 2, 2025

Copy link
Copy Markdown
ContributorAuthor

Filed #128421 for existing data races in traceback and frame inspection.

colesbury
colesbury reviewed Jan 2, 2025
View reviewed changes
Comment threadTools/tsan/suppressions_free_threading.txt Outdated
Comment threadModules/_asynciomodule.c Outdated
Comment threadModules/_asynciomodule.c Outdated
kumaraditya303
kumaraditya303 commented Jan 2, 2025
View reviewed changes
Comment threadModules/_asynciomodule.c Outdated
colesbury
colesbury reviewed Jan 2, 2025
View reviewed changes
Comment threadModules/_asynciomodule.c Outdated
colesbury
colesbury approved these changes Jan 3, 2025
View reviewed changes

@colesburycolesbury left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with two comments below

Comment threadModules/_asynciomodule.c
Comment threadTools/tsan/suppressions_free_threading.txt Outdated
kumaraditya303
kumaraditya303 commented Jan 4, 2025
View reviewed changes
Comment threadTools/tsan/suppressions_free_threading.txt Outdated
@kumaraditya303
kumaraditya303 merged commit 513a4ef into python:mainJan 4, 2025
@kumaraditya303
kumaraditya303 deleted the asyncio-tsafe branch January 4, 2025 08:48
WolframAlph pushed a commit to WolframAlph/cpython that referenced this pull request Jan 4, 2025
@kumaraditya303@WolframAlph
pythongh-128002: fix many thread safety issues in asyncio (python#128147
bdfe2ca
)
* Makes `_asyncio.Task` and `_asyncio.Future` thread-safe by adding critical sections
* Add assertions to check for thread safety checking locking of object by critical sections in internal functions
* Make `_asyncio.all_tasks` thread safe when eager tasks are used
* Add a thread safety test
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this pull request Jan 8, 2025
@kumaraditya303@srinivasreddy
pythongh-128002: fix many thread safety issues in asyncio (python#128147
4a1dabe
)
* Makes `_asyncio.Task` and `_asyncio.Future` thread-safe by adding critical sections
* Add assertions to check for thread safety checking locking of object by critical sections in internal functions
* Make `_asyncio.all_tasks` thread safe when eager tasks are used
* Add a thread safety test
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@colesburycolesburycolesbury approved these changes
@1st11st1Awaiting requested review from 1st11st1 is a code owner
@asvetlovasvetlovAwaiting requested review from asvetlovasvetlov is a code owner
@willingcwillingcAwaiting requested review from willingcwillingc is a code owner

Assignees

No one assigned

Labels

skip newstopic-asynciotopic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@kumaraditya303@colesbury