Skip to content

kde: unbounded width/height allocation in numpy and cupy backends #1287

Description

@brendancol

Describe the bug

kde() and line_density() take width and height as user-controlled ints with no upper bound. The eager numpy and cupy backends allocate np.zeros((height, width), dtype=float64) (or cupy.zeros) before any check runs.

Concrete sites in xrspatial/kde.py:

  • _run_kde_numpy line 308: np.zeros(shape, dtype=np.float64)
  • _run_kde_cupy line 314: cupy.zeros(shape, dtype=cupy.float64)
  • line_density line 706: np.zeros(shape, dtype=np.float64)

A caller passing width=1_000_000, height=1_000_000 asks for ~8 TB of float64 memory (or VRAM on the GPU path). On Linux that either OOMs the process or takes the kernel with it. On the GPU it surfaces as a raw cupy allocator error that says nothing about which parameter is too large.

Same pattern as the fixes in #1223 (rasterize), #1236 (bilateral), #1256 (morphology), #1241 (convolution), #1240 (contour), #1231 (bump), #1229 (viewshed), #1257 (glcm), #1261 (dasymetric), and #1262 (cost_distance).

The dask backends (_run_kde_dask_numpy, _run_kde_dask_cupy) build per-tile allocations lazily via da.from_delayed, so they are bounded by chunk size and do not need a separate guard.

Expected behavior

kde() and line_density() raise MemoryError before allocating, with a message that names width=/height= so the user knows which parameter to shrink.

Reproduce

from xrspatial.kde import kde
kde([0.0], [0.0], bandwidth=1.0, width=1_000_000, height=1_000_000)

Fix sketch

  1. Add a local _available_memory_bytes() helper (same pattern as convolution, morphology, bump: /proc/meminfo first, psutil fallback, 2 GB last resort).
  2. Add _check_grid_memory(rows, cols) that raises MemoryError when rows * cols * 8 exceeds 50% of available RAM.
  3. Call the guard from kde() and line_density() once grid dims are resolved, before backend dispatch. Dask paths can skip it since chunk size already bounds them.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggpuCuPy / CUDA GPU supportinput-validationInput validation and error messagesoomOut-of-memory risk with large datasets

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions