Skip to content

GH-139951: Fix major GC performance regression - #140262

Merged
markshannon merged 6 commits into
python:mainfrom
faster-cpython:dont-track-tuples
Oct 21, 2025
Merged

GH-139951: Fix major GC performance regression#140262
markshannon merged 6 commits into
python:mainfrom
faster-cpython:dont-track-tuples

Conversation

@markshannon

@markshannonmarkshannon commented Oct 17, 2025•
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Member

This PR:

  • Counts number of actually tracked objects, instead of trackable objects. This ensures that untracking tuples has the desired effect of reducing GC overhead
  • Does not track most untrackable tuples during creation. This prevents large numbers of small tuples causing excessive GCs.

For the example in the original report this makes performance on main a bit better than 3.13.

Benchmarking results show this is about neutral on performance otherwise.

@markshannon
Fix major GC performance regression
2e8d020
* Count number of actually tracked objects, instead of trackable objects. This ensures that untracking tuples has the desired effect of reducing GC overhead
* Do not track most untrackable tuples during creation. This prevents large numbers of small tuples causing execessive GCs.
@bedevere-appbedevere-appBot mentioned this pull request Oct 17, 2025
Closed
@markshannonmarkshannon added needs backport to 3.14 bugs and security fixes and removed awaiting core review labels Oct 17, 2025
eendebakpt
eendebakpt reviewed Oct 17, 2025
View reviewed changes
Comment threadObjects/tupleobject.c Outdated
sergey-miryanov
sergey-miryanov reviewed Oct 18, 2025
View reviewed changes
Comment threadPython/gc.c
sergey-miryanov
sergey-miryanov reviewed Oct 18, 2025
View reviewed changes
Comment threadInclude/internal/pycore_gc.h Outdated
sergey-miryanov
sergey-miryanov reviewed Oct 18, 2025
View reviewed changes
Comment threadPython/gc.c
efimov-mikhail
efimov-mikhail reviewed Oct 18, 2025
View reviewed changes
Comment threadInclude/internal/pycore_gc.h
sergey-miryanov
sergey-miryanov reviewed Oct 18, 2025
View reviewed changes
Comment threadPython/gc.c
@sergey-miryanov

sergey-miryanov commented Oct 18, 2025

Copy link
Copy Markdown
Contributor

I'm not sure I get what android (x86_64) fails.
I can propose two solutions:

  1. Change default value for work_to_do from -5000 to 0
  2. Or remove newly added condition
if (gcstate->work_to_do < 0) {
return;
}

@efimov-mikhailefimov-mikhail mentioned this pull request Oct 18, 2025
Merged
@mhsmith

mhsmith commented Oct 19, 2025

Copy link
Copy Markdown
Member

I'm not at all familiar with the garbage collector, but one of the ways that Android differs from the other platforms is that it runs all the test suite serially in a single process.

sergey-miryanov
sergey-miryanov reviewed Oct 21, 2025
View reviewed changes
Comment threadPython/gc.c
sergey-miryanov
sergey-miryanov approved these changes Oct 21, 2025
View reviewed changes

@sergey-miryanovsergey-miryanov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good to me, thanks!

@sergey-miryanov

sergey-miryanov commented Oct 21, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

Should we redo the benchmarks to see how the final version impacts performance? Maybe we can run it on https://github.com/faster-cpython/benchmarking-public (but I don't know how).

efimov-mikhail
efimov-mikhail approved these changes Oct 21, 2025
View reviewed changes
Comment threadInclude/internal/pycore_gc.h Outdated
Comment threadMisc/NEWS.d/next/Core_and_Builtins/2025-10-17-18-03-12.gh-issue-139951.IdwM2O.rst Outdated
Comment threadObjects/tupleobject.c
@markshannon
markshannon merged commit 0c01090 into python:mainOct 21, 2025
45 checks passed
@miss-islington-app

miss-islington-appBot commented Oct 21, 2025

Copy link
Copy Markdown

Thanks @markshannon for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 21, 2025
@markshannon@miss-islington
pythonGH-139951: Fix major GC performance regression (pythonGH-140262)
f778f01
* Count number of actually tracked objects, instead of trackable objects. This ensures that untracking tuples has the desired effect of reducing GC overhead
* Do not track most untrackable tuples during creation. This prevents large numbers of small tuples causing execessive GCs.
(cherry picked from commit 0c01090)
Co-authored-by: Mark Shannon <mark@hotpy.org>
@bedevere-app

bedevere-appBot commented Oct 21, 2025

Copy link
Copy Markdown

GH-140423 is a backport of this pull request to the 3.14 branch.

@bedevere-appbedevere-appBot removed the needs backport to 3.14 bugs and security fixes label Oct 21, 2025
@ashm-devashm-dev mentioned this pull request Oct 21, 2025
Closed
@eendebakpteendebakpt mentioned this pull request Oct 21, 2025
Closed
@efimov-mikhailefimov-mikhail mentioned this pull request Oct 22, 2025
Closed
markshannon added a commit that referenced this pull request Oct 23, 2025
@markshannon
[3.14]GH-139951: Fix major GC performance regression. Backport ofGH…
d1a434f
…-140262 (GH-140447)
* Count number of actually tracked objects, instead of trackable objects. This ensures that untracking tuples has the desired effect of reducing GC overhead
* Do not track most untrackable tuples during creation. This prevents large numbers of small tuples causing execessive GCs.
This was referenced Oct 25, 2025
Merged
Closed
nascheme added a commit to nascheme/cpython that referenced this pull request Nov 27, 2025
@nascheme
Revert GC changes to "work_to_do" logic.
fef3eed
This reverts parts of the pythonGH-140262 change. The changes that affect the
tuple untracking are left unchanged. Revert the changes to the
calculation of the increment size, based on the "work_to_do" variable.
This causes cyclic garbage to be collected more quickly. Revert also
the change to test_gc.py, which was done because the expected GC
collection was taking longer to happen.
With the tuple untrack change, the performance regression as reported by
bug pythonGH-139951 is still resolved (work_to_do changes are not required).
This was referenced Nov 27, 2025
Closed
Open
StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull request Dec 6, 2025
@markshannon@StanFromIreland
pythonGH-139951: Fix major GC performance regression (pythonGH-140262)
99c908c
* Count number of actually tracked objects, instead of trackable objects. This ensures that untracking tuples has the desired effect of reducing GC overhead
* Do not track most untrackable tuples during creation. This prevents large numbers of small tuples causing execessive GCs.
@MaxiBoetherMaxiBoether mentioned this pull request Dec 10, 2025
Closed
@naschemenascheme mentioned this pull request Dec 18, 2025
Closed
sergey-miryanov pushed a commit to sergey-miryanov/cpython that referenced this pull request Jan 24, 2026
@nascheme@sergey-miryanov
Revert GC changes to "work_to_do" logic.
a9a2252
This reverts parts of the pythonGH-140262 change. The changes that affect the
tuple untracking are left unchanged. Revert the changes to the
calculation of the increment size, based on the "work_to_do" variable.
This causes cyclic garbage to be collected more quickly. Revert also
the change to test_gc.py, which was done because the expected GC
collection was taking longer to happen.
With the tuple untrack change, the performance regression as reported by
bug pythonGH-139951 is still resolved (work_to_do changes are not required).
sergey-miryanov pushed a commit to sergey-miryanov/cpython that referenced this pull request Jan 25, 2026
@nascheme@sergey-miryanov
Revert GC changes to "work_to_do" logic.
efd2a4c
This reverts parts of the pythonGH-140262 change. The changes that affect the
tuple untracking are left unchanged. Revert the changes to the
calculation of the increment size, based on the "work_to_do" variable.
This causes cyclic garbage to be collected more quickly. Revert also
the change to test_gc.py, which was done because the expected GC
collection was taking longer to happen.
With the tuple untrack change, the performance regression as reported by
bug pythonGH-139951 is still resolved (work_to_do changes are not required).
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sergey-miryanovsergey-miryanovsergey-miryanov approved these changes
@eendebakpteendebakpteendebakpt left review comments
@efimov-mikhailefimov-mikhailefimov-mikhail approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@markshannon@sergey-miryanov@mhsmith@eendebakpt@efimov-mikhail