Skip to content

gh-117657: Add TSAN suppressions for the free-threaded build - #117736

Merged
colesbury merged 4 commits into
python:mainfrom
mpage:gh-117657-add-ft-tsan-suppressions
Apr 15, 2024
Merged

gh-117657: Add TSAN suppressions for the free-threaded build#117736
colesbury merged 4 commits into
python:mainfrom
mpage:gh-117657-add-ft-tsan-suppressions

Conversation

@mpage

@mpagempage commented Apr 10, 2024

Copy link
Copy Markdown
Contributor

Add TSAN suppressions for existing races in the free-threaded build.

Additionally, skip a few tests that take a prohibitively long amount of time (> 1 hour) when TSAN is enabled and the GIL is disabled. I'm not sure why these are so much slower with TSAN (> 1 hour) vs without it (~1 minute). perf shows that all of the time is being spent in inclusively in GC in both configurations. When TSAN is enabled ~all of the exclusive cycles are in TSAN code (e.g __tsan_read8, __tsan_read1, etc.).

I cherry-picked this on top of the nogil-integration branch and verified that the free-threading TSAN tests pass locally and in CI.

Add TSAN suppressions for existing races in the free-threaded build.
@mpage
mpage requested a review from colesburyApril 11, 2024 00:13
@mpage
mpage marked this pull request as ready for review April 11, 2024 00:13

@colesburycolesbury left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

Now that we have a list we can split up the work on fixing the data races.

Comment threadTools/tsan/supressions.txt Outdated
Comment threadTools/tsan/suppressions_free_threading.txt Outdated
@mpage

Copy link
Copy Markdown
ContributorAuthor

Now that we have a list we can split up the work on fixing the data races.

Yep! I'll create issues and link them to #117657.

@corona10corona10 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well LGTM.

Comment threadLib/test/test_weakref.py Outdated

## Free-threaded suppressions

race:_add_to_weak_set

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there GH issues open for all these suppressions?

@mpagempageApr 11, 2024

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet, working on it :) I want to provide a bit more context (i.e. sample stacktraces + repro instructions) rather than just cutting issues without context.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to start with a simple list in an omnibus issue like #117657 so that there's a single place that people can go look to track who is working on what races.

I expect many of the races to have simple fixes, but we can break off the more complex races into their own issues as needed.

The work of collecting stacktraces and repros is work that can be split up once this PR is landed and we have a way of coordinating the work.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. I threw together a quick script to massage the TSAN output from the tests into an index that we can use to look for stacktraces and tests that should reproduce the races.

@colesbury
colesbury merged commit 4783206 into python:mainApr 15, 2024
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
…ython#117736)
Additionally, reduce the iterations for a few weakref tests that would
otherwise take a prohibitively long amount of time (> 1 hour) when TSAN
is enabled and the GIL is disabled.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@mpage@colesbury@pitrou@corona10