Skip to content

gh-120321: fix thread safety of concurrently iterating over async generators - #155025

Open
kumaraditya303 wants to merge 6 commits into
python:mainfrom
kumaraditya303:asyncgen-thread-safety
Open

gh-120321: fix thread safety of concurrently iterating over async generators#155025
kumaraditya303 wants to merge 6 commits into
python:mainfrom
kumaraditya303:asyncgen-thread-safety

Conversation

@kumaraditya303

@kumaraditya303kumaraditya303 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

This PR fixes thread safety of async generators similar to #142599

Comment threadObjects/genobject.c Outdated
Comment threadObjects/genobject.c Outdated
Comment threadObjects/genobject.c
Comment threadObjects/genobject.c Outdated
Comment threadObjects/genobject.c

@mpagempage left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment threadObjects/genobject.c
@nascheme

Copy link
Copy Markdown
Member

Some feedback from a review session using Claude Opus. There are some edge cases that would be good to fix:

  • The claim on the async generator should only have one owner. proposed fix. This adds a new state "RUNNING" and renames the "ITER" state to "SUSPENDED". This seems like the most important one in terms of thread-safety.
  • Use CAS when closing the awaitable on a finished frame. commit
  • Close athrow awaitable when the operation completes. commit
  • Validate the athrow() argument before claiming the generator. commit
  • Document that a completed athrow() awaitable cannot be reused. commit

Full branch with commits on top of this PR branch. I'm not too familiar with the genobject.c code so the changes in that branch definitely need a second review before merging as-is.

I also found an additional thing but I'll make a new issue: logic related to ag_hooks_inited appears to be racey.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@kumaraditya303@nascheme@mpage