(perf): remove out of core median in normalize_total

[Intron7]

Computing counts per cell for normalize_total leads to better performance.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.51%. Comparing base (f0f6960) to head (e2888c0).
Report is 40 commits behind head on main.

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3379   +/-   ##
=======================================
  Coverage   75.50%   75.51%           
=======================================
  Files         113      113           
  Lines       13140    13141    +1     
=======================================
+ Hits         9922     9924    +2     
+ Misses       3218     3217    -1     

Files with missing lines	Coverage Δ
src/scanpy/preprocessing/_normalization.py	82.02% <100.00%> (+1.34%)	⬆️

[flying-sheep]

Fair enough. Can this blow up? I.e. is counts ever big enough that this is a problem?

[Intron7]

So I did some testing with 1Billion cells. 1Billion cells at 64bit are roughly 8GB. The median costs another 8GB. Thats roughly 16GB. I dont think 16GB of Ram is an issue for someone that analyses 1Billion cells.

[ilan-gold]

Why is this for the 1.12 milestone?

[flying-sheep]

probably because no 1.11.1 milestone existed for a bit (my fault, I’ll be cleaner about this in the future)

[ilan-gold]

Please add a test to get the coverage to pass, but the code itself looks fine.

[flying-sheep]

Looks like the tests don’t pass anymore. Any idea what’s wrong?

[selmanozleyen]

They passed the same tests on the PR right? The results I checked were due to small differences I am not sure why

[flying-sheep]

Ah, it’s unrelated. The test that failed doesn’t call normalize_total.