Add Index.should_add_coord_to_array

[benbovy]

• ClosesDataArray: propagate index coordinates with non-array dimensions #10116
• Tests added
• User visible changes (including notable bug fixes) are documented in whats-new.rst
• New functions/methods are listed in api.rst

Same goal than #10116 using the alternative approach suggested in #10116 (comment) where the propagation (validation) of coordinates in a DataArray is delegated to their index (if any).

I find this approach cleaner than #10116. Here is the same notebook example adapted for this PR.

Index API alternatives

1. Index.validate_dataarray_coord(self, name: Hashable, var: Variable, dims: set[Hashable]) -> None

   • check one index coordinate at a time, thus called multiple times for a multi-coordinate index

2. Index.validate_dataarray_coords(self, variables: dict[Hashable, Variable], dims: set[Hashable]) -> None

   • check all coordinates of an index at once
   • perhaps could make things easier for developers of custom Xarray indexes (perhaps not?), but it would make Xarray internals more complicated with possible significant impact on performance (we'd have to rely on Indexes.group_by_index while in option 1 we simply need to iterate over _variables, _coords and/or _indexes).

3. Index.validate_dataarray_coords(self, variables: dict[Hashable, Variable], dims: set[Hashable]) -> Coordinates

   • Same than option 2 but allow returning a new set of coordinate variables and their indexes.
   • This has much more flexibility (e.g., convert a custom 2D spatial index to a 1D default PandasIndex when the result has only one of the two spatial dimensions), but I find it also complicated and prone to confusion.

Option 1, the simplest and working well with IntervalIndex, is currently implemented in this PR.

cc @shoyer@dcherian

[shoyer]

Some quick thoughts

[benbovy]

Thanks @shoyer for taking a look!

This is now ready for (another round of) review.

Compared to #10116 this PR still represents a major data model change for DataArray (in the sense that it allows overriding strict enforcement of the DataArray model in certain cases), but I think that the risk is mitigated since it is here done explicitly via Index.validate_dataarray_coord and the default behavior is to keep conforming to the DataArray model. Limiting an index to (in)validate its coordinates may also be good to keep things simple and predictable for end-users.

@dcherian regarding your #10116 (comment), in the example notebook the DataArray reprs shown without any list of dimensions for the Coordinates section do not strike me much. That said, the Coordinates repr is still basic and might benefit from such addition (it doesn't have any html repr BTW).

[dcherian]

Nice!

[dcherian]

That said, the Coordinates repr is still basic and might benefit from such addition (it doesn't have any html repr BTW).

Yeah I think it could be improved, but certainly not a blocker here.

IMO we can merge. I'm not a huge fan of the method name validate_dataarray_coord but I couldn't come up with anything better

[dcherian]

Yeah this method's a hard one to name. The only thing that comes to mind is should_propagate_variable_with_dataarray :/

[benbovy]

Another possible name: is_coord_needed_in_dataarray()

[dcherian]

I added suggestions for should_add_coord_to_array to save a few characters