(feat): Support for pandasExtensionArray

[ilan-gold]

Some outstanding points/decisions brought up by this PR:

• Confirm type promotion rules and write them out. As it stands now, if everything is of the same extension array type, it is passed onwards and otherwise is converted to numpy. (related: Avoid coercing to numpy in as_shared_dtypes #8714)
  - [ ] Acceptance of plum as a dispatch method. Without it, the behavior should be fallen back on from before (cast to numpy types). I am a big fan of dispatching and think it could serve as a model going forward for making support of other data types/arrays more feasible. The other option, I think, would be to just use the underlying array of the ExtensionDuckArray class to decide and then have some central registry that serves as the basis for a decorator (like the api for accessors via _CachedAccessor). That being said, the current defaults are quite good so this is a marginal feature, in all likelihood.
• Do we allow just pandas ExtensionArray directly or can we also allow Series?

Possible missing something else! Let me know!

Checklist:

• ClosesCategorical Array #8463 and ClosesSupport for pandas Extension Arrays #5287
• Tests added
• User visible changes (including notable bug fixes) are documented in whats-new.rst
• New functions/methods are listed in api.rst

[welcome]

Thank you for opening this pull request! It may take us a few days to respond here, so thank you for being patient.
If you have questions, some answers may be found in our contributing guidelines.

[TomNicholas]

This looks like a big feature addition!

I'm very ignorant of ExtensionArrays - but is it possible to imagine a design where ExtensionDuckArray conforms well enough to the duck array structure that xarray expects that ExtensionDuckArray could live entirely outside of xarray? I'm also curious why we need isinstance(data, ExtensionDuckArray rather than calling our already-existing is_duck_array(data).

[ilan-gold]

@TomNicholas That is an interesting point. Once you pare back the ExtensionDuckArray business and just look at the remaining code, it largely has to do with dtype checking. And checking something like isinstance(self.variables[k].data, ExtensionDuckArray) could easily be done (I believe) with pd.api.extensions.is_extension_array_dtype(self.variables[k].data). So we could just tell people "xarray will support pandasExtensionArraydtype classes so long as they conform to NEP18." As for why we don't do is_duck_array, most of the type checking I have added occurs at spots where we just don't want numpy to touch our array class, like scale/offset, type promoting, or instantiation of a new Variable. I didn't even run into is_duck_array from what I remember, probably exactly because this class passes those checks as it should.

My problem with removing ExtensionDuckArray from the core is that someone (maybe xarray) is going to make ExtensionDuckArray available for hacking on gist or something, everyone will use it (including probably us for testing), and then it's a quasi-public API instead of being part of the core. I am generally not a fan of this sort of thing, but it appears I am in the minority in the open-source community. In this case, though, NEP18 for ExtensionArray classes could come at some point which would render this class not-needed, although it's not public facing anyway so the change would not be a breaking one.

[ilan-gold]

What's the status here?

[dcherian]

Sorry, this fell off my radar. Can you open an issue regarding the to_dataframe problem?

[welcome]

Congratulations on completing your first pull request! Welcome to Xarray! We are proud of you, and hope to see you again!