Skip to content

Add memory guard to viewshed numpy path #1229

Description

@brendancol

Describe the bug

_viewshed_cpu in xrspatial/viewshed.py allocates several arrays sized from the raster dimensions with no memory check. Peak working memory is around 500 bytes per pixel once you count the sort buffer, so a 20000x20000 raster (400M pixels) tries to allocate ~200 GB and either OOM-kills or thrashes swap.

The dask path already has a guard — see _viewshed_dask around line 2172: it checks r2_bytes against _available_memory_bytes() and raises MemoryError with a hint to use max_distance. The numpy path has nothing.

To Reproduce

import numpy as np
import xarray as xr
from xrspatial import viewshed

# Tries to allocate a multi-hundred-GB event_list before any check
arr = np.zeros((20000, 20000), dtype=np.float64)
xs = np.arange(20000, dtype=float)
ys = np.arange(20000, dtype=float)
r = xr.DataArray(arr, coords=dict(x=xs, y=ys), dims=["y", "x"])
viewshed(r, x=10000.0, y=10000.0, observer_elev=5)

Expected behavior

MemoryError before allocation, with a message pointing users at max_distance=.

Proposed fix

Guard at the top of _viewshed_cpu: estimate peak working memory (~500 * H * W bytes) via _available_memory_bytes() and raise if it would blow past a safe fraction of RAM.

Allocations that add up

  • event_list: 3 * H * W * 7 * 8 = 168 bytes/pixel
  • status_values + status_struct + idle: ~104 bytes/pixel
  • visibility_grid: 8 bytes/pixel
  • lexsort temporary: ~2x event_list during the sort

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinginput-validationInput validation and error messagesoomOut-of-memory risk with large datasets

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions