Performance Shift(s): `d0569c79`

[github-actions]

Benchmark comparison has identified performance shifts at commit d0569c7 (#4620). Please review the report below and take corrective/congratulatory action as appropriate 🙂

Performance shift report

       before           after         ratio
     [b42c65da]       [d0569c79]
     <main~2>         <main~1>  
-        0.234375       0.13671875     0.58  netcdf_save.NetcdfSave.track_addedmem_netcdf_save(1, True)

Generated by GHA run 1936268272

[trexfeathers]

@pp-mo has recently suggested we revisit the memory benchmarks as they throw up false positives quite often. Needs some more thought - I'm not sure if it's possible to have a threshold override for specific benchmarks in ASV?

[jamesp]

Why are memory benchmarks erratic? I naiively would have expected them more deterministic than timing benchmarks.

[trexfeathers]

Why are memory benchmarks erratic? I naiively would have expected them more deterministic than timing benchmarks.

@jamesp these are a home-brew version that attempt to estimate the extra memory a process has added. ASV by default is only able to measure the peak memory usage, and as you say it can get very consistent results. But we need to measure added memory because even a sizeable shift in memory demand might be dwarfed by an even larger baseline - 'flying under the radar' of the 1.2 ratio threshold used in our checks.

[trexfeathers]

Discussion with @pp-mo: unfortunately our home-brew added memory estimation looks particularly vulnerable to noise, so I have removed the smallest size parameter for our memory monitoring benchamarks. I've included this in the existing #4621 as working on main will generate conflicts. 73c0f86

[pp-mo]

#4621 addresses