gh-149584: Fix excessive overhead in the Tachyon profiler regarding the cache behavior

[pablogsal]

Some ideas after the discussion in the issue with @maurycy. The profiler was spending too much time on repeated remote-memory bookkeeping, full remote page reads for small fixed-size structs, repeated remote writes of unchanged frame-cache state, and Python object allocation churn on steady-state frame-cache hits.

This PR improves the profiler by:

• reading hot interpreter/thread/frame structs with exact remote reads instead of page-cache reads
• tracking the number of live page-cache entries so cache clear/search only scans the used prefix
• batching predicted interpreter/thread/frame reads with process_vm_readv() on Linux
• reusing cached FrameInfo and thread id objects when frame-cache hits dominate

The last_profiled_frame remote-write suppression is already present in current upstream/main, so this branch keeps that baseline behavior and builds on top of it.

Benchmark

Benchmarked with:

./python Tools/inspection/benchmark_external_inspection.py --duration 3

For the per-commit measurements, I used the same benchmark workload in quiet mode with cache_frames=True and all_threads=True.

Step	Work rate	Sample rate	Avg call time	Incremental win
upstream/main baseline	382,593/s	363,963/s	2.614 us	-
Exact reads for hot structs	549,741/s	514,642/s	1.819 us	+167,148/s (+43.7%)
Track live page-cache entries	747,946/s	688,282/s	1.337 us	+198,205/s (+36.1%)
Batch predicted remote reads	897,703/s	806,084/s	1.114 us	+149,756/s (+20.0%)
Reuse profiler result objects	1,187,314/s	1,047,698/s	0.842 us	+289,611/s (+32.3%)

Final benchmark using the benchmark script:

Metric	Baseline	Final	Change
Work rate	382,593/s	1,180,434/s	3.08x faster
Sample rate	363,963/s	1,038,965/s	2.85x faster
Avg call time	2.614 us	0.85 us	-67.5%
Success rate	99.67%	99.84%	+0.17 pp

Final benchmark output:

Average call time: 0.85 us
Work rate: 1,180,434.2 calls/sec
Sample rate: 1,038,964.9 samples/sec
Success rate: 99.84%

• Issue: _remote_debugging: reading whole pages over and over #149584

[pablogsal]

@maurycy do you mind reviewing this PR?

[maurycy]

@pablogsal Awesome job, I'm delighted :) Definitely will do!

Perhaps it's worth an entry in the "Optimizations" section?

One thing that I'm wondering immediately is extending benchmark_external_inspection with get_async_stack_trace or even get_all_awaited_by.

[pablogsal]

One thing that I'm wondering immediately is extending benchmark_external_inspection with get_async_stack_trace or even get_all_awaited_by.

Added async benchmark modes so the benchmark harness can exercise get_stack_trace(), get_async_stack_trace(), and get_all_awaited_by().

[pablogsal]

Perhaps it's worth an entry in the "Optimizations" section?

Since is beta1 I think we are good for now. Later makes sense

[pablogsal]

Can you give it another go?

[maurycy]

@pablogsal Left two more comments:

• gh-149584: Fix excessive overhead in the Tachyon profiler regarding the cache behavior #149649 (comment)
• gh-149584: Fix excessive overhead in the Tachyon profiler regarding the cache behavior #149649 (comment)

(If they do not show, you have to check the resolved threads; the comments are follow-ups.)

[pablogsal]

Will take a look in 1h

[maurycy]

@pablogsal One last thing (in the current round): after v. before numbers for the newly added async benchmarks in the description, to see the gain and for future reference (for git blame tourists, like me)

[pablogsal]

I'm traveling so I don't have access to the same box where I run the original ones so I will need to do these in a different one :)

[maurycy]

I'm traveling so I don't have access to the same box where I run the original ones so I will need to do these in a different one :)

Thank you. Happy PyCon! :)

[pablogsal]

@maurycy addressed the comments

[maurycy]

Given:

typedefstruct {
uintptr_tinterpreter_addr;
uintptr_tthread_state_addr;
uint64_tcode_object_generation;
} InterpreterThreadCacheEntry;

https://github.com/python/cpython/pull/149649/changes#diff-a6ce0bef7825b17047f05fdec7aa610bc1d2d2b368e72b6871a79c3b1d2dfaf9R242-R246

Can code_object_generation become stale here?

[maurycy]

Given:

typedefstruct {
uintptr_tinterpreter_addr;
uintptr_tthread_state_addr;
uint64_tcode_object_generation;
} InterpreterThreadCacheEntry;

https://github.com/python/cpython/pull/149649/changes#diff-a6ce0bef7825b17047f05fdec7aa610bc1d2d2b368e72b6871a79c3b1d2dfaf9R242-R246

Can thread_state_addr become stale here?

I've added these two "mirror" comments because I'm puzzled about the lifecycle of InterpreterThreadCacheEntry.

[pablogsal]

Oh sorry, let me push a fix. The intended design is that each writer fully owns its cache slot: the same pattern the other shortcut already uses, where tstate and generation each have their own (interpreter_addr, value) pair. The L2 table joins both values under one slot, which is what makes the lifecycle hard to reason about. I'll split InterpreterThreadCacheEntry into two single-purpose tables, InterpreterTstateCacheEntryand InterpreterGenerationCacheEntry, so the L2 matches the L1 design and each writer fully owns its table.

[maurycy]

But this is also per interpreter, or not?

[pablogsal]

This is notactually per-interpreter: code_object_cache is a single hashtable on the unwinder, keyed by the remote code object's address. It's shared across all interpreters in the target process.

[maurycy]

@pablogsal Thank you. 🙏🏼 I've left three comments. The complexity gets real. 😞 Bump about the new benchmark numbers. Random idea is improving test coverage with many interpreters, similarly to:

cpython/Lib/test/test_external_inspection.py

Lines 3381 to 3452 in a8e3247

	deftest_cache_vs_no_cache_equivalence(self):
	"""Test that cache_frames=True and cache_frames=False produce equivalent results."""
	script_body="""\
	def level3():
	sock.sendall(b"ready"); sock.recv(16)
	def level2():
	level3()
	def level1():
	level2()
	level1()
	"""
	withself._target_process(script_body) as (
	p,
	client_socket,
	make_unwinder,
	):
	_wait_for_signal(client_socket, b"ready")
	# Sample with cache
	unwinder_cache=make_unwinder(cache_frames=True)
	frames_cached=self._get_frames_with_retry(
	unwinder_cache, {"level1", "level2", "level3"}
	)
	# Sample without cache
	unwinder_no_cache=make_unwinder(cache_frames=False)
	frames_no_cache=self._get_frames_with_retry(
	unwinder_no_cache, {"level1", "level2", "level3"}
	)
	client_socket.sendall(b"done")
	self.assertIsNotNone(frames_cached)
	self.assertIsNotNone(frames_no_cache)
	# Same number of frames
	self.assertEqual(len(frames_cached), len(frames_no_cache))
	# Same function names in same order
	funcs_cached= [f.funcnameforfinframes_cached]
	funcs_no_cache= [f.funcnameforfinframes_no_cache]
	self.assertEqual(funcs_cached, funcs_no_cache)
	# For level3 (leaf frame), due to timing races we can be at either
	# sock.sendall() or sock.recv() - both are valid. For parent frames,
	# the locations should match exactly.
	# Valid locations for level3: line 3 has two statements
	# sock.sendall(b"ready") -> col 4-26
	# sock.recv(16) -> col 28-41
	level3_valid_cols= {(4, 26), (28, 41)}
	foriinrange(len(frames_cached)):
	loc_cached=frames_cached[i].location
	loc_no_cache=frames_no_cache[i].location
	ifframes_cached[i].funcname=="level3":
	# Leaf frame: can be at either statement
	self.assertIn(
	(loc_cached.col_offset, loc_cached.end_col_offset),
	level3_valid_cols,
	)
	self.assertIn(
	(loc_no_cache.col_offset, loc_no_cache.end_col_offset),
	level3_valid_cols,
	)
	else:
	# Parent frames: must match exactly
	self.assertEqual(loc_cached, loc_no_cache)

[pablogsal]

@pablogsal Thank you. 🙏🏼 I've left three comments. The complexity gets real. 😞 Bump about the new benchmark numbers.

I won't have access to my box until next week :(

Random idea is improving test coverage with many interpreters, similarly to:

I don't think this helps a lot. I tried out of curiosity and this test passes without any of the bugfixes we have added. We can try to add more tests in a separate PR if you want :)

[pablogsal]

Measured in a different machine:

get_async_stack_trace() — baseline vs final

Metric	Baseline	Final	Change
Work rate	101,117/s	160,804/s	1.59x faster
Sample rate	97,857/s	153,686/s	1.57x faster
Avg call time	9.89 µs	6.22 µs	-37.1%
Success rate	98.93%	98.90%	-0.03 pp

get_all_awaited_by() — baseline vs final

Metric	Baseline	Final	Change
Work rate	19,241/s	22,103/s	1.15x faster
Sample rate	19,157/s	21,973/s	1.15x faster
Avg call time	51.97 µs	45.24 µs	-12.9%
Success rate	100.00%	100.00%	flat

[maurycy]

@pablogsal Thank you. It's delightful work 🔥, and I appreciate the opportunity. I approve!

[miss-islington-app]

Thanks @pablogsal for the PR 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

[bedevere-app]

GH-150152 is a backport of this pull request to the 3.15 branch.

[pablogsal]

Thanks for the great review @maurycy. The code is much better thanks to you 🖤