pstats - allow stats sorting by cumulative time per call and total time per call

[alexnvdias]

BPO	18795
Nosy	@birkenfeld, @bitdancer, @jstasiak, @DanielNoord
PRs	bpo-18795: Add cumpercall and totalpercall keys to pstats sort_stats #31919
Files	pstats.py: pstats source code with patch added pstats-diff-issue18795.txt: Diff of patch against current version of pstats (Aug 21 2013)

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2013-08-20.23:48:11.951>
labels = ['type-feature', 'library']
title = 'pstats - allow stats sorting by cumulative time per call and total time per call'
updated_at = <Date 2022-03-15.23:29:12.822>
user = 'https://bugs.python.org/alexnvdias'

bugs.python.org fields:

activity = <Date 2022-03-15.23:29:12.822>
actor = 'danielnoord'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2013-08-20.23:48:11.951>
creator = 'alexnvdias'
dependencies = []
files = ['31391', '31396']
hgrepos = []
issue_num = 18795
keywords = ['patch']
message_count = 9.0
messages = ['195731', '195762', '195763', '195767', '195819', '195821', '196753', '209544', '415305']
nosy_count = 7.0
nosy_names = ['georg.brandl', 'r.david.murray', 'python-dev', 'alexnvdias', 'jstasiak', 'ramnes', 'danielnoord']
pr_nums = ['31919']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue18795'
versions = ['Python 3.4']

[alexnvdias]

Me and a couple of colleagues have stumbled upon the need to sometimes sort our profiling stats by time (cumulative and total) per call. I believe this could be useful to other people, and have therefore opened this patch.

Thanks for taking the time to look into it.

-Alexandre

[bitdancer]

I believe the module already supports sorting by time and cumulative time (determining the top consumers of cpu is a major part of the point of the profile module, after all).

What exactly is the change that you are proposing? If you post a diff instead of a modified copy of the module it would be easier to see what you are proposing.

[alexnvdias]

It does support sorting by total time and cumulative time spent on a function, but not by the time per each call of that function.

I've uploaded a diff of the patch.

-Alexandre

[bitdancer]

It would be clearer, I think, to call it average time. I must admit to not being sure why that is useful. I'm also worried there might be backward compatibility issues with changing the ordering of the data structure. If new fields are going to be added they should probably be added at the end.

[alexnvdias]

Average time would indeed be a better description.

As for its usefulness: let's say that for example you would want to find the function in which your code spends the most time on average, in order to later optimise said function. If one of the calls to another function has its execution time spike momentarily, it will be at the top of the total time column on the result. However, that does not necessarily mean that it was the function in which on average the most time was spent on, and thus the one that the optimisation effort should be focused on.
Also, when seeing the profiling results, I would expect to be able to sort by any of the columns shown, and that is currently not possible for the average cumulative and total time per call.

Do you mind clarifying your worries with backwards compatibility? Unless I'm mistaken, "fcn_list "is the only outside effect of the method I'm changing. It is built from the last element of each tuple in stats_list, whose ordering is what I'm proposing to be able to change (thus changing the ordering of the function list, but that is already done by other sort options).

[bitdancer]

There may not be a problem. I'm not familiar enough with the pstats code to review the patch for real without some study.

[jstasiak]

I'd change cumulativepercall and totalpercall to cumpercall and percall (and/or intpercall) but that's a detail, this patch works for me.

Data structure affected by this patch is produced and consumed internally by the sort method so it looks like nothing changes from the point of view of client code.

[alexnvdias]

Could I get an update on this please?

[DanielNoord]

I resubmitted the original patch and updated it with documentation and news item changes.

This would still be a valuable addition imo, as it can help single out expensive calls in your module/script.

[DanielNoord]

Gentle ping on this issue now that the Github migration has retagged some users that might potentially be interested in this. The PR only needs a core review, which I know are in high demand... Would be wonderful to get this in 3.12 😄