Skip to content

(fix): ExtensionArray + DataArray roundtrip - #9520

Merged
dcherian merged 8 commits into
pydata:mainfrom
ilan-gold:ig/fix_extension_array_dataarray_roundtrip
Sep 21, 2024
Merged

(fix): ExtensionArray + DataArray roundtrip#9520
dcherian merged 8 commits into
pydata:mainfrom
ilan-gold:ig/fix_extension_array_dataarray_roundtrip

Conversation

@ilan-gold

Copy link
Copy Markdown
Contributor

I drive-by added name to the Series since it wasn't there before.

@ilan-gold ilan-gold changed the title (fix): fix extension array + dataarray roundtrip (fix): ExtensionArray + DataArray roundtrip Sep 19, 2024
Comment thread xarray/core/variable.py Outdated
Comment thread xarray/core/dataarray.py Outdated
Comment thread properties/test_pandas_roundtrip.py
@dcherian dcherian added the plan to merge Final call for comments label Sep 19, 2024
@dcherian
dcherian merged commit e649e13 into pydata:main Sep 21, 2024
@ilan-gold
ilan-gold deleted the ig/fix_extension_array_dataarray_roundtrip branch September 23, 2024 10:21
hollymandel pushed a commit to hollymandel/xarray that referenced this pull request Sep 23, 2024
* (fix): fix extension array + dataarray roundtrip

* (fix): satisfy mypy

* (refactor): move check out of `Variable.values`

* (fix): ensure `mypy` is happy with `values` typing

* (fix): setter with `mypy`

* (fix): remove case of `values`
@shoyer

shoyer commented Oct 1, 2024

Copy link
Copy Markdown
Member

It appears that this PR may have broken some upstream pandas tests, specifically testing round-trips with various index types:
https://github.com/pandas-dev/pandas/blob/e78ebd3f845c086af1d71c0604701ec49df97228/pandas/tests/generic/test_to_xarray.py#L32

Here's a minimal test case:

import pandas as pd
import numpy as np

cat = pd.Categorical(list("abcd"))
df = pd.DataFrame({"f": cat}, index=cat)
restored = df.to_xarray().to_dataframe()
print(restored.index)  # Index(['a', 'b', 'c', 'd'], dtype='object', name='index')
print(df.index)  # CategoricalIndex(['a', 'b', 'c', 'd'], categories=['a', 'b', 'c', 'd'], ordered=False, dtype='category')

I'm not sure if this is a pandas or xarray issue, but it's one or the other!

(My guess is that most of these tests in pandas should probably live in xarray instead, given that we implement all the conversion logic.)

@ilan-gold

Copy link
Copy Markdown
Contributor Author

@shoyer This is probably an "us" issue. I will look into it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

plan to merge Final call for comments

Projects

None yet

Development

Successfully merging this pull request may close these issues.

to_pandas on DataArray with extension array data type

3 participants