gh-145685: per-type method cache implementation

[kumaraditya303]

• Issue: Unexpected scaling bottleneck in _PyType_Lookup / _PyType_LookupStackRefAndVersion #145685

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @kumaraditya303 for commit 728021c 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F150160%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

[colesbury]

A few more comments below (mostly found by Claude). Also:

• Have you run single-threaded benchmarks to ensure that we haven't introduced any regressions with this?
• Does the type_lookup benchmark scale as well as the other scaling benchmarks? In other words, should we consider this a partial or full fix for the type cache scaling issues?

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #33682891 | 📁 Comparing 42a6ae8 against main (8646385)

🔍 Preview build

155 files changed · + 3 added · ± 151 modified · - 1 deleted

+Added

• library/asyncio-tools.html
• library/tkinter.fontchooser.html
• library/tkinter.systray.html

±Modified

• glossary.html
• py-modindex.html
• c-api/bytes.html
• c-api/dict.html
• c-api/exceptions.html
• c-api/extension-modules.html
• c-api/float.html
• c-api/frame.html
• c-api/gen.html
• c-api/import.html
• and 141 more...

-Deleted

• library/asyncio-policy.html

[kumaraditya303]

Have you run single-threaded benchmarks to ensure that we haven't introduced any regressions with this?

Benchmark results show that it is 1% slower on gil enabled build https://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20260616-3.16.0a0-4fbec93/bm-20260616-vultr-x86_64-kumaraditya303-mrocache-3.16.0a0-4fbec93-vs-base.md

I think it it is within the error of benchmark machines and not significant.

[markshannon]

I'm very wary of dismissing 1% slower as "within the noise" without some other evidence that there is no slowdown for a change like this.

It also looks like you've missed several of the stats.
How do the stats look for cache hits, misses and collisions?

How does this change memory use?

[colesbury]

Results from running locally on my machine (via Claude, GIL-enabled, clang-20, optimized, lto=thin) look good: https://gist.github.com/colesbury/1e5c1eb4ccc1e626696b568d18d8bbc4. Cache misses drop substantially, but the overall effect is small (possibly within the noise margin) because:

• the MRO cache doesn't matter as much now that we rely heavily on specialization.
• the MRO hit rate was already high on these benchmarks

A few more suggestions:

• We should drop the type_cache_collisions stat because it's not longer tracked and doesn't make sense in the same way. Previously a collision in the MRO cache would overwrite the previous cache entry; now we do linear probing.
• We should track the number of times we resize/clear cache a type's cache

[kumaraditya303]

We should track the number of times we resize/clear cache a type's cache

I've added stats for resizes and invalidations and removed collisions.

[kumaraditya303]

@markshannon Do you have more comments/concerns on this?

[dpdani]

I would still like to see the top comment "Lock-free per type method cache implementation" rephrased since it's not a lock free implementation.

[kumaraditya303]

I would still like to see the top comment "Lock-free per type method cache implementation" rephrased since it's not a lock free implementation.

How would you rephrase it, do you have a suggestion?

[dpdani]

How about just "Per-type method cache implementation"?