Description
surface_distance(), surface_allocation(), and surface_direction() on the numpy and cupy backends have no memory guard. _dijkstra (line 113) and _dijkstra_geodesic (line 176) allocate h_keys / h_rows / h_cols arrays of size height * width plus a visited mask. _init_arrays (line 233) allocates four full-size arrays (dist, alloc, src_row, src_col). _surface_distance_cupy (lines 448-451) does the same on the GPU.
Per-pixel cost on numpy is roughly 80 bytes: dist+alloc float64 (16), src_row+src_col int64 (16), visited int8 (1), h_keys+h_rows+h_cols (24), output float32 (4), direction-mode temps (16). CuPy backend is ~72 bytes/pixel on the GPU.
A 50000x50000 raster asks for ~200 GB of working memory before anything errors out, and the process gets OOM-killed.
Dask paths (_surface_distance_dask_bounded via map_overlap, and _sd_dask_iterative per-tile) are bounded by chunk size and inherit the guard via per-chunk numpy calls.
cost_distance.py already has _check_memory(height, width) and _check_gpu_memory(height, width) helpers that raise MemoryError before allocation. surface_distance already imports _heap_push / _heap_pop from cost_distance but never calls those guards.
Same asymmetric-guard pattern fixed in sky_view_factor #1299, sieve #1296, kde #1287, resample #1295, focal #1284, geodesic #1283, mahalanobis #1288, true_color #1291, diffuse #1267, erode #1275, emerging_hotspots #1274, and dasymetric #1261.
Expected behavior
surface_distance, surface_allocation, and surface_direction raise MemoryError with a clear message on the eager numpy and cupy backends when the projected working set exceeds available memory. Dask paths skip the guard.
Proposed fix
Add _available_memory_bytes(), _available_gpu_memory_bytes(), _check_memory(rows, cols), and _check_gpu_memory(rows, cols) helpers (80 B/pixel for numpy, 72 B/pixel for cupy, 50% threshold). Call them from _surface_distance_numpy and _surface_distance_cupy before heap and output allocations.
Description
surface_distance(),surface_allocation(), andsurface_direction()on the numpy and cupy backends have no memory guard._dijkstra(line 113) and_dijkstra_geodesic(line 176) allocateh_keys/h_rows/h_colsarrays of sizeheight * widthplus avisitedmask._init_arrays(line 233) allocates four full-size arrays (dist,alloc,src_row,src_col)._surface_distance_cupy(lines 448-451) does the same on the GPU.Per-pixel cost on numpy is roughly 80 bytes: dist+alloc float64 (16), src_row+src_col int64 (16), visited int8 (1), h_keys+h_rows+h_cols (24), output float32 (4), direction-mode temps (16). CuPy backend is ~72 bytes/pixel on the GPU.
A 50000x50000 raster asks for ~200 GB of working memory before anything errors out, and the process gets OOM-killed.
Dask paths (
_surface_distance_dask_boundedviamap_overlap, and_sd_dask_iterativeper-tile) are bounded by chunk size and inherit the guard via per-chunk numpy calls.cost_distance.pyalready has_check_memory(height, width)and_check_gpu_memory(height, width)helpers that raiseMemoryErrorbefore allocation.surface_distancealready imports_heap_push/_heap_popfromcost_distancebut never calls those guards.Same asymmetric-guard pattern fixed in sky_view_factor #1299, sieve #1296, kde #1287, resample #1295, focal #1284, geodesic #1283, mahalanobis #1288, true_color #1291, diffuse #1267, erode #1275, emerging_hotspots #1274, and dasymetric #1261.
Expected behavior
surface_distance,surface_allocation, andsurface_directionraiseMemoryErrorwith a clear message on the eager numpy and cupy backends when the projected working set exceeds available memory. Dask paths skip the guard.Proposed fix
Add
_available_memory_bytes(),_available_gpu_memory_bytes(),_check_memory(rows, cols), and_check_gpu_memory(rows, cols)helpers (80 B/pixel for numpy, 72 B/pixel for cupy, 50% threshold). Call them from_surface_distance_numpyand_surface_distance_cupybefore heap and output allocations.