The streaming fallback in reproject() allocates its output as a 2-D buffer (result = np.full(out_shape, nodata, dtype=np.float64)), but the per-tile worker (_reproject_chunk_numpy) returns (rows, cols, bands) tiles for 3-D sources. Assembly then fails:
ValueError: could not broadcast input array from shape (16,16,3) into shape (16,16)
Both assembly loops in _reproject_streaming (the dask.bag branch and the local ThreadPoolExecutor branch) have the same problem.
Repro (direct call, since the end-to-end trigger needs a >512 MB source and no dask):
import importlib
import numpy as np
import xarray as xr
rp = importlib.import_module('xrspatial.reproject')
from xrspatial.reproject._crs_utils import _resolve_crs
from xrspatial.reproject._grid import _compute_output_grid
raster = xr.DataArray(
np.random.rand(32, 32, 3), dims=['y', 'x', 'band'],
coords={'y': np.linspace(55, 45, 32), 'x': np.linspace(-5, 5, 32), 'band': [1, 2, 3]},
attrs={'crs': 'EPSG:4326', 'nodata': np.nan},
)
src_crs = _resolve_crs('EPSG:4326')
tgt_crs = _resolve_crs('EPSG:32633')
src_bounds = rp._source_bounds(raster)
grid = _compute_output_grid(src_bounds, (32, 32), src_crs, tgt_crs)
rp._reproject_streaming(
raster, src_bounds, (32, 32), True,
src_crs.to_wkt(), tgt_crs.to_wkt(),
grid['bounds'], grid['shape'],
'bilinear', float('nan'), 16,
16, rp._parse_max_memory('1GB'),
x_desc=False, band_nodata=None,
)
To hit it end to end: reproject a multi-band raster over 512 MB in an environment without dask. reproject() sets _use_streaming = True and hands the 3-D source straight to _reproject_streaming.
Expected: a (out_h, out_w, bands) result, matching the in-memory and dask paths. The band-axis handling those paths got in #2027 never reached the streaming fallback.
Found while adding test coverage for the streaming path. A strict xfail in that test PR pins this crash until it's fixed.
The streaming fallback in
reproject()allocates its output as a 2-D buffer (result = np.full(out_shape, nodata, dtype=np.float64)), but the per-tile worker (_reproject_chunk_numpy) returns(rows, cols, bands)tiles for 3-D sources. Assembly then fails:Both assembly loops in
_reproject_streaming(the dask.bag branch and the local ThreadPoolExecutor branch) have the same problem.Repro (direct call, since the end-to-end trigger needs a >512 MB source and no dask):
To hit it end to end: reproject a multi-band raster over 512 MB in an environment without dask.
reproject()sets_use_streaming = Trueand hands the 3-D source straight to_reproject_streaming.Expected: a
(out_h, out_w, bands)result, matching the in-memory and dask paths. The band-axis handling those paths got in #2027 never reached the streaming fallback.Found while adding test coverage for the streaming path. A strict xfail in that test PR pins this crash until it's fixed.