Uh oh!
There was an error while loading. Please reload this page.
(fix): pandas extension array repr for int64[pyarrow] - #10317
Conversation
ilan-gold
commented
May 13, 2025
Separately, I am not sure if |
ilan-gold
commented
May 13, 2025
Reverted the addition of |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
ilan-gold
commented
May 14, 2025
Is it possible these three tests are flaky? It seems to pass locally for me and on the face of it, has nothing to do with this PR> |
kmuehlbauer
commented
May 14, 2025
@ilan-gold This surfaced with the release of dask 2025.5.0. Something has changed on the dask side and/or on the xarray side, which let the error surface. Here is the stacktrace on my machine: xarray/tests/test_backends.py:827 (TestDask.test_outer_indexing_reversed)
self=<xarray.tests.test_backends.TestDaskobjectat0x7f2da44c2ff0>deftest_outer_indexing_reversed(self) ->None:
# regression test for GH6560ds=xr.Dataset(
{"z": (("t", "p", "y", "x"), np.ones((1, 1, 31, 40)))},
)
withself.roundtrip(ds) ason_disk:
>subset=on_disk.isel(t=[0], p=0).z[:, ::10, ::10][:, ::-1, :]
/home/kai/python/projects/xarray/xarray/tests/test_backends.py:835: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/kai/python/projects/xarray/xarray/core/dataset.py:2764: iniselvar=var.isel(var_indexers)
/home/kai/python/projects/xarray/xarray/core/variable.py:1032: iniselreturnself[key]
/home/kai/python/projects/xarray/xarray/core/variable.py:782: in__getitem__returnself._finalize_indexing_result(dims, data)
/home/kai/python/projects/xarray/xarray/core/variable.py:786: in_finalize_indexing_resultreturnself._replace(dims=dims, data=data)
/home/kai/python/projects/xarray/xarray/core/variable.py:938: in_replacereturntype(self)(dims, data, attrs, encoding, fastpath=True)
/home/kai/python/projects/xarray/xarray/core/variable.py:365: in__init__super().__init__(
/home/kai/python/projects/xarray/xarray/namedarray/core.py:264: in__init__self._dims=self._parse_dimensions(dims)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self=<[AttributeError("'Variable' object has no attribute '_dims'") raisedinrepr()] Variableobjectat0x7f2d5c6c6380>dims= ('t', 'y', 'x')
def_parse_dimensions(self, dims: _DimsLike) ->_Dims:
dims= (dims,) ifisinstance(dims, str) elsetuple(dims)
iflen(dims) !=self.ndim:
>raiseValueError(
f"dimensions {dims} must have the same length as the "f"number of data dimensions, ndim={self.ndim}"
)
EValueError: dimensions ('t', 'y', 'x') musthavethesamelengthasthenumberofdatadimensions, ndim=4/home/kai/python/projects/xarray/xarray/namedarray/core.py:508: ValueError |
ilan-gold
commented
May 14, 2025
@kmuehlbauer Thanks for the heads up, sounds like I need to update my env then. Thanks for the info here! |
kmuehlbauer
commented
May 14, 2025
See new issue #10321. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
* main: Fix performance regression in interp from pydata#9881 (pydata#10370) html repr: improve style for dropdown sections (pydata#10354) Grouper tweaks. (pydata#10362) Docs: Add links to getting help mermaid diagram (pydata#10324) Enforce ruff/flynt rules (FLY) (pydata#10375) Add missing AbstractWritableDataStore base methods and arguments (pydata#10343) Improve html repr in dark mode (Jupyterlab + Xarray docs) (pydata#10353) Pin Mypy to 1.15 (pydata#10378) use numpy dtype exposed by zarr array instead of metadata.data_type (pydata#10348) Fix doc typo for caption "Interoperability" (pydata#10374) Implement cftime vectorization as discussed in PR pydata#8322 (pydata#8324) Enforce ruff/flake8-pyi rules (PYI) (pydata#10359) Apply assorted ruff/Pylint rules (PL) / Enforce PLE rules (pydata#10366) (fix): pandas extension array repr for int64[pyarrow] (pydata#10317) Enforce ruff/flake8-implicit-str-concat rules (ISC) (pydata#10368) Enforce ruff/refurb rules (FURB) (pydata#10367) Ignore ruff/Pyflakes rule F401 more precisely (pydata#10369) Apply assorted ruff/flake8-simplify rules (SIM) (pydata#10364) Apply assorted ruff/flake8-pytest-style rules (PT) (pydata#10363) Fix "a array" misspelling (pydata#10365)
Similar to #10315, we uncover more unexpected behavior with
int64[pyarrow]as the testing data type.We were apparently relying on a call to
np.array(via thexp.reshape) internally for the actual printing of the elements but this is not implemented forint64[pyarrow](unlike other extension array types) and therefore broke without an explicit implementation. So we add that implementationOnce this has been handled, the
PandasExtensionArrayis no longer being converted into a numpy array silently, and thus we need to ensure that the repr can handle thePandasExtensionArraynatively.Lastly we add a
KeyErrorto the lookup to ensure that this sort of thing cannot happen again.From (fix): no
fill_valueon reindex #10304Tests added
User visible changes (including notable bug fixes) are documented in
whats-new.rstNew functions/methods are listed in
api.rst