Implement a streaming executor with prefetch and a memory budget

[fishidaho]

This is the single largest gap between the current repository and the project's actual thesis. There is no out-of-core execution model anywhere in src/vsparse/ today. load_and_normalize fully decodes the entire index stream into RAM before doing anything with it, so its peak memory scales with total dataset size rather than a bounded working set. On a large cohort this can run to tens of GiB.

Proposed scope

• A chunked execution model with prefetch: src/vsparse/ gains a real streaming path, built and tested as library code (not a one-off script).
• A user-settable memory budget that bounds the working set regardless of dataset size.
• Double-buffered prefetch so chunk decode overlaps with kernel execution.
• Every terminal reduction (sum, mean, variance, per-cell/per-gene statistics) reimplemented as a streaming accumulator over chunks, rather than requiring the full decoded array.

Acceptance criteria

• Peak RSS is flat as dataset size grows across at least two orders of magnitude, enforced in CI a benchmarking memory-ceiling infrastructure.
• Numerical agreement with the in-memory reference path matches a tight relative-error tolerance on X @ Omega, B @ X, and norm computations.
• The memory budget parameter is documented and actually enforced, not advisory.

References

• src/vsparse/_rapid_load.py (load_and_normalize).

[aarmey]

I guess this is a little different from the scope I have always envisioned... VCSR/VCSC cut the in-memory size by ~50% relative to a csr_array with no loss in accuracy. Consequently, you can load more data onto RAM/vRAM than you otherwise would be able to.

However, I do see the point that we are also providing wrappers such that the data never needs to be modified in memory. Given how rapid this makes loading from disk, it's an intriguing idea that you never actually load the full dataset ever. I think I'd want to see benchmarking to understand the performance implications. I know there's streaming to the GPU, but I'd also want to understand how that would work.

[fishidaho]

@aarmey I've only done a preliminary audit with the IBD dataset performing these operations on a parafac2 fit, and the speed regression seemed minimal. I would obviously have to benchmark on many more cases with a clean Aretha to see what is actually the benefit to having it. Could be a good weekend task.