Skip to content

_reproject_streaming crashes on 3-D (multi-band) sources #3100

Description

@brendancol

The streaming fallback in reproject() allocates its output as a 2-D buffer (result = np.full(out_shape, nodata, dtype=np.float64)), but the per-tile worker (_reproject_chunk_numpy) returns (rows, cols, bands) tiles for 3-D sources. Assembly then fails:

ValueError: could not broadcast input array from shape (16,16,3) into shape (16,16)

Both assembly loops in _reproject_streaming (the dask.bag branch and the local ThreadPoolExecutor branch) have the same problem.

Repro (direct call, since the end-to-end trigger needs a >512 MB source and no dask):

import importlib
import numpy as np
import xarray as xr
rp = importlib.import_module('xrspatial.reproject')
from xrspatial.reproject._crs_utils import _resolve_crs
from xrspatial.reproject._grid import _compute_output_grid

raster = xr.DataArray(
    np.random.rand(32, 32, 3), dims=['y', 'x', 'band'],
    coords={'y': np.linspace(55, 45, 32), 'x': np.linspace(-5, 5, 32), 'band': [1, 2, 3]},
    attrs={'crs': 'EPSG:4326', 'nodata': np.nan},
)
src_crs = _resolve_crs('EPSG:4326')
tgt_crs = _resolve_crs('EPSG:32633')
src_bounds = rp._source_bounds(raster)
grid = _compute_output_grid(src_bounds, (32, 32), src_crs, tgt_crs)
rp._reproject_streaming(
    raster, src_bounds, (32, 32), True,
    src_crs.to_wkt(), tgt_crs.to_wkt(),
    grid['bounds'], grid['shape'],
    'bilinear', float('nan'), 16,
    16, rp._parse_max_memory('1GB'),
    x_desc=False, band_nodata=None,
)

To hit it end to end: reproject a multi-band raster over 512 MB in an environment without dask. reproject() sets _use_streaming = True and hands the 3-D source straight to _reproject_streaming.

Expected: a (out_h, out_w, bands) result, matching the in-memory and dask paths. The band-axis handling those paths got in #2027 never reached the streaming fallback.

Found while adding test coverage for the streaming path. A strict xfail in that test PR pins this crash until it's fixed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions