feat: add opt-in rasterization tiles

[stanbot8]

Preview. Text in square brackets becomes links in the submission.

Two implementations exist. This one uses 2 commits: the same 1st commit as PR 1167, then an opt-in tiling commit. The other implementation uses 1 output canvas and only removes the label-count limit. Compare the limitations.

Summary

Add optional output tiles to rasterization. tile_size=None keeps the current behavior. A positive tile_size limits each spatial output chunk. It also limits each Datashader canvas for points and shapes. This branch fixes the label-count failure in [issue 987]. Closes issue 987.

Design

Images and labels pass bounded output chunks to the affine transform. Points use half-open internal tile bounds. Shapes use a 1-pixel halo and crop it. One spatial index selects the shapes for each tile. Sorted positions preserve Datashader first semantics.

The change is in [rasterize.py].

Benchmark

The benchmark rasterized a 3 × 4096 × 4096float32 image. The output size was 192 MiB. Each result was stored in a memory-mapped file with single-threaded Dask. The table shows medians from 7 runs on an Intel Core i7-8700K.

tile_size	Largest output chunk	Graph tasks	Median time	Time change	Peak RSS increase	RSS reduction
None	192 MiB	257	4.400 s	—	398.2 MiB	—
1024	12 MiB	756	4.009 s	-8.9%	221.2 MiB	44.5%
512	3 MiB	1220	3.682 s	-16.3%	216.2 MiB	45.7%
256	0.75 MiB	2628	4.500 s	+2.3%	218.5 MiB	45.1%

For this workload, tile_size=1024 reduced the peak RSS increase by 44.5%.

The image benchmark source and recorded results use candidate aab46d8ec96acb9213a51cd244397a12a2f0579f.

An indexed-shape benchmark used 4096 shapes and 64 tiles. The index reduced the Datashader input from 262145 to 5042 rows (98.1%). It reduced the largest tile input from 4096 to 81 shapes (98.0%). The median time fell from 1.443 s to 0.957 s (33.7%). Both outputs had the same checksum. See the shape benchmark source and recorded results.

Benchmark commands

uv run python benchmarks/benchmark_rasterize_tiling.py --size 4096 --repeats 7 --tile-size 0uv run python benchmarks/benchmark_rasterize_tiling.py --size 4096 --repeats 7 --tile-size 1024uv run python benchmarks/benchmark_rasterize_tiling.py --size 4096 --repeats 7 --tile-size 512uv run python benchmarks/benchmark_rasterize_tiling.py --size 4096 --repeats 7 --tile-size 256

Limitations

Smaller tiles create more graph tasks. A shape that crosses tile boundaries is rendered in each intersecting tile.

Tests

Raster tests verify bounded chunks for 2D images, 2D labels, and 3D multiscale images. Point tests cover internal tile edges, global maximum bounds, and uneven tile sizes. Shape tests cover overlaps and categorical channels. Each tiled result is identical to the untiled result.

The rasterization test file passes with 14 tests. The complete suite passes with 1371 tests, 7 skips, and 1 expected failure.

The tests are in [test_rasterize.py].

Commands

python -m pytest tests/core/operations/test_rasterize.py -qpython -m pytest --cov --color=no -n auto --dist worksteal -qruff check src/spatialdata/_core/operations/rasterize.py tests/core/operations/test_rasterize.pyruff format --check src/spatialdata/_core/operations/rasterize.py tests/core/operations/test_rasterize.py