Skip to content

BUG: Fix display with nested NumPy arrays - #10222

Merged
dcherian merged 6 commits into
pydata:mainfrom
basnijholt:bug-formatting
Jun 20, 2025
Merged

BUG: Fix display with nested NumPy arrays#10222
dcherian merged 6 commits into
pydata:mainfrom
basnijholt:bug-formatting

Conversation

@basnijholt

@basnijholtbasnijholt commented Apr 11, 2025

Copy link
Copy Markdown
Contributor

Currently, the following will fail to display:

importxarrayasxrimportnumpyasnpx=np.empty((2, 2), dtype=object)
foriinrange(2):
forjinrange(2):
x[i, j] =np.zeros(2) # Set to 1D array of size 2ds=xr.DataArray(x)
ds
Click to see the error message
---------------------------------------------------------------------------ValueErrorTraceback (mostrecentcalllast)
File~/Work/pipefunc/.venv/lib/python3.13/site-packages/IPython/core/formatters.py:347, inBaseFormatter.__call__(self, obj)
345method=get_real_method(obj, self.print_method)
346ifmethodisnotNone:
-->347returnmethod()
348returnNone349else:
File~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/common.py:188, inAbstractArray._repr_html_(self)
186ifOPTIONS["display_style"] =="text":
187returnf"<pre>{escape(repr(self))}</pre>"-->188returnformatting_html.array_repr(self)
File~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting_html.py:323, inarray_repr(arr)
315arr_name=f"'{arr.name}'"ifgetattr(arr, "name", None) else""317header_components= [
318f"<div class='xr-obj-type'>{obj_type}</div>",
319f"<div class='xr-array-name'>{arr_name}</div>",
320format_dims(dims, indexed_dims),
321 ]
-->323sections= [array_section(arr)]
325ifhasattr(arr, "coords"):
326sections.append(coord_section(arr.coords))
File~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting_html.py:232, inarray_section(obj)
226collapsed= (
227"checked"228if_get_boolean_with_default("display_expand_data", default=True)
229else""230 )
231variable=getattr(obj, "variable", obj)
-->232preview=escape(inline_variable_array_repr(variable, max_width=70))
233data_repr=short_data_repr_html(obj)
234data_icon=_icon("icon-database")
File~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting.py:304, ininline_variable_array_repr(var, max_width)
302returnvar._data._repr_inline_(max_width)
303ifgetattr(var, "_in_memory", False):
-->304returnformat_array_flat(var, max_width)
305dask_array_type=array_type("dask")
306ifisinstance(var._data, dask_array_type):
File~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting.py:223, informat_array_flat(array, max_width)
220# every item will take up at least two characters, but we always want to221# print at least first and last items222max_possibly_relevant=min(max(array.size, 1), max(math.ceil(max_width/2.0), 2))
-->223relevant_front_items=format_items(
224first_n_items(array, (max_possibly_relevant+1) //2)
225 )
226relevant_back_items=format_items(last_n_items(array, max_possibly_relevant//2))
227# interleave relevant front and back items:228# [a, b, c] and [y, z] -> [a, z, b, y, c]File~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting.py:212, informat_items(x)
209elifnp.logical_not(time_needed).all():
210timedelta_format="date"-->212formatted= [format_item(xi, timedelta_format) forxiinx]
213returnformattedFile~/Work/pipefunc/.venv/lib/python3.13/site-packages/xarray/core/formatting.py:193, informat_item(x, timedelta_format, quote_strings)
191returnrepr(x) ifquote_stringselsex192elifhasattr(x, "dtype") andnp.issubdtype(x.dtype, np.floating):
-->193returnf"{x.item():.4}"194else:
195returnstr(x)
ValueError: canonlyconvertanarrayofsize1toaPythonscalar

Whenever there are size==1 arrays, it currently does work:

importxarrayasxrimportnumpyasnpx=np.empty((2, 2), dtype=object)
foriinrange(2):
forjinrange(2):
x[i, j] =np.zeros((1, 1, 1)) # Set to 3D array of size 1ds=xr.DataArray(x)
ds

Should I add these examples as regression tests anywhere, if so, where?

For context, I am running into this issue in https://github.com/pipefunc/pipefunc where one can put the resulting objects into xarray.Datasets.
For example, see the docs here https://pipefunc.readthedocs.io/en/latest/examples/physics-simulation/ (search for xarray.Dataset).

image
  • Closes #xxxx
  • Tests added
  • User visible changes (including notable bug fixes) are documented in whats-new.rst
  • New functions/methods are listed in api.rst

@welcome

welcomeBot commented Apr 11, 2025

Copy link
Copy Markdown

Thank you for opening this pull request! It may take us a few days to respond here, so thank you for being patient.
If you have questions, some answers may be found in our contributing guidelines.

Currently, the following will fail to display:
```python
import xarray as xr
import numpy as np
x = np.empty((2, 2), dtype=object)
for i in range(2):
for j in range(2):
x[i, j] = np.zeros(2) # Set to 1D array of size 2
ds = xr.DataArray(x)
ds
```
Whenever there are `size==1` arrays, it currently does work:
```python
import xarray as xr
import numpy as np
x = np.empty((2, 2), dtype=object)
for i in range(2):
for j in range(2):
x[i, j] = np.zeros((1, 1, 1)) # Set to 3D array of size 1
ds = xr.DataArray(x)
ds
For context, I am running into this issue in https://github.com/pipefunc/pipefunc where one can put the resulting objects into `xarray.Dataset`s.
For example, see the docs here https://pipefunc.readthedocs.io/en/latest/examples/physics-simulation/ (search for `xarray.Dataset`).
Comment threadxarray/tests/test_formatting.py Outdated
@basnijholt

Copy link
Copy Markdown
ContributorAuthor

Now test_slice_in_title_single_item_array is failing with:

 - d = [10.009]
? - ^^^
+ d = 10.01
? ^

This test was introduced in #5948 and touches the exact same code that I changed. I think the test that is failing now is wrong and should be updated to d = [10.009] instead.

@basnijholt
basnijholt requested a review from dcherianJune 19, 2025 00:45
@dcherian

Copy link
Copy Markdown
Contributor

OK please update the test in that case. Merging in main should fix the other "array api" failures

@basnijholt

Copy link
Copy Markdown
ContributorAuthor

@dcherian, I updated the tests, all is passing except some (I believe) unrelated failure.

@dcherian
dcherian merged commit 99a1ad2 into pydata:mainJun 20, 2025
@welcome

welcomeBot commented Jun 20, 2025

Copy link
Copy Markdown

Congratulations on completing your first pull request! Welcome to Xarray! We are proud of you, and hope to see you again! celebration gif

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@basnijholt@dcherian