gh-108303: Move `Lib/test/sortperf.py` to `Tools/scripts` by sobolevn · Pull Request #114687 · python/cpython

sobolevn · 2024-01-28T20:18:58Z

@corona10 what do you think about moving these case to pyperformance? Would it be benefitial?

Issue: Move test files into test subdirectories #108303

vstinner · 2024-01-29T15:25:49Z

Which change (who) created this script? Is it @corona10?

sobolevn · 2024-01-29T15:35:27Z

It was Guido 27 years ago: https://github.com/python/cpython/commits/main/Lib/test/sortperf.py

I think he agreed on moving it away in Discord's #python-dev channel.

corona10

IMO, the First thing we have to do is add the benchmark script, not remove the file.

bedevere-app · 2024-01-29T17:20:09Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

sobolevn · 2024-01-29T19:04:32Z

@corona10 ok, I will send a PR to pyperformance soon! 👍

This reverts commit 6903baa.

corona10

Even if we just move the script into Tools/script
IMO pyperf based script is more modernized way and easy to compare with improvement patch. WDYT?

sobolevn · 2024-02-09T14:37:32Z

I agree with Guido: it is not really that useful as a benchmark (since it is a micro-benchmark). It is only interesting when working on list.sort directly. And I don't think that this is going to happen anytime soon, since the sorting algorithm is quite old, fast, and stable.

So, I think that we can move this file for now. If someone has a real use-case for list.sort benchmark in pyperf, they can recreate my existing closed PR.

And for now this file would serve more like a history reference.

corona10 · 2024-02-10T06:04:03Z

So, I think that we can move this file for now. If someone has a real use-case for list.sort benchmark in pyperf, they can recreate my existing closed PR.

Ah, I meant rewriting the benchmark script based on pyperf(you already did) and storing it Tools/script/, I did not mean to move new script to pyperformance, I also agree with Guido, it will not be a proper place.

sobolevn · 2024-02-10T09:28:59Z

@corona10 sure!

corona10 · 2024-02-11T12:28:05Z

I will take a look at the PR by tomorrow :)

corona10

By default, all benchmark should be executed.
Benchmarks needs some tune up

.....................
list_sort: Mean +- std dev: 30.8 ms +- 0.4 ms
.....................
list_sort_ascending: Mean +- std dev: 1.76 ms +- 0.10 ms

Execution time is too long :(
(Maybe we need to control inner_loops value)

sobolevn · 2024-02-15T09:05:41Z

@corona10 I've merged your suggestions. Thank you!

I've also optimized the benchmark:

» time python Tools/scripts/sortperf.py .....................
list_sort: Mean +- std dev: 29.0 us +- 0.2 us
.....................
list_sort_ascending: Mean +- std dev: 28.9 us +- 0.4 us
.....................
list_sort_ascending_exchanged: Mean +- std dev: 29.2 us +- 0.4 us
.....................
list_sort_ascending_one_percent: Mean +- std dev: 29.5 us +- 0.2 us
.....................
list_sort_ascending_random: Mean +- std dev: 29.6 us +- 0.3 us
.....................
list_sort_descending: Mean +- std dev: 29.5 us +- 0.2 us
.....................
list_sort_duplicates: Mean +- std dev: 30.4 us +- 1.1 us
.....................
list_sort_equal: Mean +- std dev: 29.2 us +- 0.2 us
.....................
list_sort_worst_case: Mean +- std dev: 29.2 us +- 0.2 us
python Tools/scripts/sortperf.py 98.38s user 3.21s system 99% cpu 1:41.75 total

Is it good enough?

corona10

LGTM

@sobolevn Feel free to merge the PR anytime!

sobolevn · 2024-02-18T07:25:11Z

@corona10 Thank you! 🎉

I think that we also need to backport this to 3.11 and 3.12
Do you agree?

corona10 · 2024-02-18T07:25:49Z

Yeah go a head

miss-islington-app · 2024-02-18T07:27:18Z

Thanks @sobolevn for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

…onGH-114687) (cherry picked from commit f9154f8) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

bedevere-app · 2024-02-18T07:27:27Z

GH-115625 is a backport of this pull request to the 3.12 branch.

…onGH-114687) (cherry picked from commit f9154f8) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

bedevere-app · 2024-02-18T07:27:32Z

GH-115626 is a backport of this pull request to the 3.11 branch.

…114687) (#115626) gh-108303: Move `Lib/test/sortperf.py` to `Tools/scripts` (GH-114687) (cherry picked from commit f9154f8) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

…114687) (#115625) gh-108303: Move `Lib/test/sortperf.py` to `Tools/scripts` (GH-114687) (cherry picked from commit f9154f8) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

…on#114687)

pythongh-108303: RemoveLib/test/sortperf.py
6903baa

sobolevn requested a review from vstinner January 28, 2024 20:18

bedevere-appBot added the awaiting review label Jan 28, 2024

bedevere-appBot mentioned this pull request Jan 28, 2024
Move test files into test subdirectories #108303
Closed

sobolevn added the skip news label Jan 28, 2024

corona10 requested changes Jan 29, 2024
View reviewed changes

bedevere-appBot added awaiting changes and removed awaiting review labels Jan 29, 2024

sobolevn mentioned this pull request Feb 7, 2024
Add list.sort benchmarks python/pyperformance#328
Closed

sobolevn added 2 commits February 9, 2024 14:56

Revert "pythongh-108303: RemoveLib/test/sortperf.py"
145f089
This reverts commit 6903baa.

Move Lib/test/sortperf.py to Tools/scripts
3b38525

sobolevn changed the title ~~gh-108303: Remove Lib/test/sortperf.py~~gh-108303: Move Lib/test/sortperf.py to Tools/scriptsFeb 9, 2024

sobolevn requested a review from corona10 February 9, 2024 11:57

corona10 reviewed Feb 9, 2024
View reviewed changes

sobolevn added 3 commits February 10, 2024 15:17

Use pyperf script
979bca0

Fix CI
052ec30

Merge branch 'main' into issue-108303-sortperf
273eed8

corona10 requested changes Feb 12, 2024
View reviewed changes

Comment threadTools/scripts/sortperf.py Outdated
Comment threadTools/scripts/sortperf.py Outdated
Comment threadTools/scripts/sortperf.py Outdated

Merge branch 'main' into issue-108303-sortperf
6c549d6

Address review
7c2dd81

corona10 approved these changes Feb 17, 2024
View reviewed changes

bedevere-appBot added awaiting merge and removed awaiting changes labels Feb 17, 2024

sobolevn added needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Feb 18, 2024

sobolevn merged commit f9154f8 into python:mainFeb 18, 2024

bedevere-appBot removed the awaiting merge label Feb 18, 2024

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Feb 18, 2024
pythongh-108303: MoveLib/test/sortperf.pytoTools/scripts(pyth…
f52145e
…onGH-114687) (cherry picked from commit f9154f8) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

bedevere-appBot removed the needs backport to 3.12 only security fixes label Feb 18, 2024

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Feb 18, 2024
pythongh-108303: MoveLib/test/sortperf.pytoTools/scripts(pyth…
c846be2
…onGH-114687) (cherry picked from commit f9154f8) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

bedevere-appBot removed the needs backport to 3.11 only security fixes label Feb 18, 2024

woodruffw pushed a commit to woodruffw-forks/cpython that referenced this pull request Mar 4, 2024
pythongh-108303: MoveLib/test/sortperf.pytoTools/scripts(pyth…
67b6939
…on#114687)

pochmann3 mentioned this pull request Mar 10, 2024
sortperf.py is broken #116576
Closed

sobolevn mentioned this pull request Mar 10, 2024
gh-116576: Fix Tools/scripts/sortperf.py sorting the same list #116577
Merged

diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
pythongh-108303: MoveLib/test/sortperf.pytoTools/scripts(pyth…
3f47339
…on#114687)

LukasWoodtli pushed a commit to LukasWoodtli/cpython that referenced this pull request Jan 22, 2025
pythongh-108303: MoveLib/test/sortperf.pytoTools/scripts(pyth…
4fb68a3
…on#114687)

Uh oh!

Uh oh!

Conversation

sobolevn commented Jan 28, 2024• edited by bedevere-app BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented Jan 29, 2024

Uh oh!

sobolevn commented Jan 29, 2024

Uh oh!

corona10 left a comment

Choose a reason for hiding this comment

Uh oh!

bedevere-appBot commented Jan 29, 2024

Uh oh!

sobolevn commented Jan 29, 2024

Uh oh!

corona10 left a comment

Choose a reason for hiding this comment

Uh oh!

sobolevn commented Feb 9, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

corona10 commented Feb 10, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sobolevn commented Feb 10, 2024

Uh oh!

corona10 commented Feb 11, 2024

Uh oh!

corona10 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sobolevn commented Feb 15, 2024

Uh oh!

corona10 left a comment

Choose a reason for hiding this comment

Uh oh!

sobolevn commented Feb 18, 2024

Uh oh!

corona10 commented Feb 18, 2024

Uh oh!

miss-islington-appBot commented Feb 18, 2024

Uh oh!

bedevere-appBot commented Feb 18, 2024

Uh oh!

bedevere-appBot commented Feb 18, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sobolevn commented Jan 28, 2024•
edited by bedevere-app Bot
Loading

sobolevn commented Feb 9, 2024•
edited
Loading

corona10 commented Feb 10, 2024•
edited
Loading

corona10 left a comment •
edited
Loading