Skip to content

gh-149718: Aggregate same stack frames in Tachyon in some collectors - #149719

Merged
pablogsal merged 7 commits into
python:mainfrom
maurycy:remote-debugger-batching
May 12, 2026
Merged

gh-149718: Aggregate same stack frames in Tachyon in some collectors#149719
pablogsal merged 7 commits into
python:mainfrom
maurycy:remote-debugger-batching

Conversation

@maurycy

@maurycymaurycy commented May 12, 2026

Copy link
Copy Markdown
Contributor

Please see gh-149718 for more details.

On a MacBook M4, using test_asyncio since it has longer stacks (without --async-aware):

Sample rate (samples/sec)

Formatmain (c6fd7de)remote-debugger-batching (2864c275847)
--pstats31,161.59103,657.01
--collapsed57,302.45102,350.22
--flamegraph33,490.73100,641.14
--gecko31,346.3798,031.88
--heatmap14,960.6497,607.28
--jsonl24,554.5698,018.93

Error rate (%)

Formatmain (c6fd7de)remote-debugger-batching (2864c275847)
--pstats1.940.54
--collapsed1.470.78
--flamegraph1.770.57
--gecko1.920.57
--heatmap2.330.44
--jsonl2.060.54

Missed samples (%)

Formatmain (c6fd7de)remote-debugger-batching (2864c275847)
--pstats90.6568.90
--collapsed82.8169.29
--flamegraph89.9569.81
--gecko90.6070.59
--heatmap95.5170.72
--jsonl92.6370.59

Bench

mkdir -p /tmp/bench_results
git checkout c6fd7de
sudo -n ./python.exe -m profiling.sampling run -r 300khz --pstats -o /dev/null -m test test_asyncio 2>&1| tee /tmp/bench_results/main_pstats.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --collapsed -o /dev/null -m test test_asyncio 2>&1| tee /tmp/bench_results/main_collapsed.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --flamegraph -o /dev/null -m test test_asyncio 2>&1| tee /tmp/bench_results/main_flamegraph.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --gecko -o /dev/null -m test test_asyncio 2>&1| tee /tmp/bench_results/main_gecko.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --heatmap -o /tmp/heatmap_main -m test test_asyncio 2>&1| tee /tmp/bench_results/main_heatmap.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --jsonl -o /dev/null -m test test_asyncio 2>&1| tee /tmp/bench_results/main_jsonl.txt
git checkout remote-debugger-batching
sudo -n ./python.exe -m profiling.sampling run -r 300khz --pstats -o /dev/null -m test test_asyncio 2>&1| tee /tmp/bench_results/batching_pstats.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --collapsed -o /dev/null -m test test_asyncio 2>&1| tee /tmp/bench_results/batching_collapsed.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --flamegraph -o /dev/null -m test test_asyncio 2>&1| tee /tmp/bench_results/batching_flamegraph.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --gecko -o /dev/null -m test test_asyncio 2>&1| tee /tmp/bench_results/batching_gecko.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --heatmap -o /tmp/heatmap_batching -m test test_asyncio 2>&1| tee /tmp/bench_results/batching_heatmap.txt
sudo -n ./python.exe -m profiling.sampling run -r 300khz --jsonl -o /dev/null -m test test_asyncio 2>&1| tee /tmp/bench_results/batching_jsonl.txt

Comment threadLib/profiling/sampling/collector.py

@pablogsalpablogsal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks fantastic! Thanks a lot @maurycy 🚀 Excellent job

@pablogsal
pablogsal enabled auto-merge (squash) May 12, 2026 23:31
@pablogsalpablogsal added the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label May 12, 2026
@pablogsal
pablogsal merged commit 76f2285 into python:mainMay 12, 2026
58 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @maurycy for the PR, and @pablogsal for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

@bedevere-app

Copy link
Copy Markdown

GH-149747 is a backport of this pull request to the 3.15 branch.

@bedevere-appbedevere-appBot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label May 12, 2026
@maurycy
maurycy deleted the remote-debugger-batching branch May 12, 2026 23:47
pablogsal pushed a commit that referenced this pull request May 13, 2026
…ectors (GH-149719) (#149747)
gh-149718: Aggregate same stack frames in Tachyon in some collectors (GH-149719)
(cherry picked from commit 76f2285)
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@maurycy@pablogsal