support for additional scipy nd interpolants by hollymandel · Pull Request #9599 · pydata/xarray

hollymandel · 2024-10-09T18:15:24Z

Closesfollow upstream scipy interpolation improvements #7704
Adds support for n dimensional tensor product interpolants slinear, cubic, quintic, and pchip.
Updates documentation of DataArray/Dataset.interp and DataArray/Dataset.interp_like to reflect updates to scipy interpolants called.
Adds argument reduce to allow disabling of reduction of interpolation along independent dimensions (see DataArray.interp() : poor performance #2223). Prior to this the behavior was always reduce=True. However for certain n-dimensional interpolants this will change the mathematical behavior of the interpolation (I think), so I have added an option for users to turn this off.

Outstanding uncertainty: dask/chunking behavior? Nothing about this PR touches the chunking of interpolation but I am a bit worried about this. Many interpolants, including those previously supported in one dimension, require nearby/all points from the original coordinate, and also don't parallelize well over the new coordinate. I'm not quite sure how this is currently being handled.

headtr1ck · 2024-10-13T17:32:57Z

Thanks for this contribution.

Some remarks:

What is the minimal scipy version that is required for these new interpolators? Is it aligned with the minimum version for xarray or do we need some checks for that?

You have added several formatting changes to totally unrelated parts that make it more difficult to review then necessary. Please try to avoid that.

Doc build failed, have to check if random or related to your docstrings.

hollymandel · 2024-10-14T15:20:07Z

Thanks for the comments!

What is the minimal scipy version that is required for these new interpolators? Is it aligned with the minimum version for xarray or do we need some checks for that?

I'm seeing 1.11 in ci/requirements/min-all-deps.yml and these interpolants were added in 1.10 so I think we're good.

You have added several formatting changes to totally unrelated parts that make it more difficult to review then necessary. Please try to avoid that.

Oops, reverted, sorry!

headtr1ck · 2024-10-14T16:08:39Z

Also, I think the name "reduce" is not very intuitive, but I fail to come up with an alternative. "decompose" seems weird out of context. Does anyone have a suggestion?

hollymandel · 2024-10-17T20:45:25Z

Also, I think the name "reduce" is not very intuitive, but I fail to come up with an alternative. "decompose" seems weird out of context. Does anyone have a suggestion?

Perhaps the _interp makes decompose_interp sound less weird? Separate or split also sound OK to me.

dcherian · 2024-10-21T21:11:25Z

Many interpolants, including those previously supported in one dimension, require nearby/all points from the original coordinate, and also don't parallelize well over the new coordinate. I'm not quite sure how this is currently being handled.

I think we end up rechunking to a single in a very roundabout way. I don't think you need to be too concerned about it for this PR. It does need an overhaul though

…ot be 1d decomposed

Co-authored-by: Deepak Cherian <dcherian@users.noreply.github.com>

hollymandel · 2024-11-04T03:53:39Z

I think the failed test in ubuntu-latest py3.12 flaky is unrelated to this PR. I've rerun a few times and it keeps happening though, not sure how to move forward.

keewis · 2024-11-04T09:39:09Z

that issue should be fixed by #9709, so the merge appears to have fixed that issue here, as well.

* main: Enforce ruff/flake8-pie rules (PIE) (pydata#9740) Enforce ruff/flake8-comprehensions rules (C4) (pydata#9724) Enforce ruff/Perflint rules (PERF) (pydata#9730) Apply ruff rule RUF007 (pydata#9739) chmod -x (pydata#9725) Aplpy ruff rules (RUF) (pydata#9731) Fix typos found by codespell (pydata#9721) support for additional scipy nd interpolants (pydata#9599) Apply ruff/flake8-simplify rules (SIM) (pydata#9727) Apply ruff/flake8-implicit-str-concat rules (ISC) (pydata#9722) Apply ruff/flake8-pie rules (PIE) (pydata#9726) Enforce ruff/pygrep-hooks rules (PGH) (pydata#9729) Move to micromamba 2 (pydata#9732) Fix groupby tests (pydata#9716) Add missing xarray.core.missing import (pydata#9714) Fix writing of DataTree subgroups to zarr or netCDF (pydata#9677) Bump pypa/gh-action-pypi-publish from 1.10.3 to 1.11.0 in the actions group (pydata#9707) Update pre-commit hooks (pydata#9713)

* main: (125 commits) http:// → https:// (pydata#9748) Discard useless `!s` conversion in f-string (pydata#9752) Apply ruff/flake8-simplify rule SIM401 (pydata#9749) Use micromamba 1.5.10 where conda is needed (pydata#9737) pin array-api-strict<=2.1 (pydata#9751) Reorganise ruff rules (pydata#9738) use new conda-forge package pydap-server (pydata#9741) Enforce ruff/flake8-pie rules (PIE) (pydata#9740) Enforce ruff/flake8-comprehensions rules (C4) (pydata#9724) Enforce ruff/Perflint rules (PERF) (pydata#9730) Apply ruff rule RUF007 (pydata#9739) chmod -x (pydata#9725) Aplpy ruff rules (RUF) (pydata#9731) Fix typos found by codespell (pydata#9721) support for additional scipy nd interpolants (pydata#9599) Apply ruff/flake8-simplify rules (SIM) (pydata#9727) Apply ruff/flake8-implicit-str-concat rules (ISC) (pydata#9722) Apply ruff/flake8-pie rules (PIE) (pydata#9726) Enforce ruff/pygrep-hooks rules (PGH) (pydata#9729) Move to micromamba 2 (pydata#9732) ...

hollymandel force-pushed the issue_7704 branch 8 times, most recently from b8e4c50 to 34fb497Compare October 11, 2024 19:09

hollymandel marked this pull request as ready for review October 11, 2024 19:14

hollymandel force-pushed the issue_7704 branch from 8568ca4 to 1def6d0Compare October 12, 2024 02:14

hollymandel force-pushed the issue_7704 branch 2 times, most recently from 4a26a7c to 34fb497Compare October 14, 2024 15:18

hollymandel force-pushed the issue_7704 branch 2 times, most recently from 33f52ec to 211984dCompare October 14, 2024 15:49

hollymandel force-pushed the issue_7704 branch 2 times, most recently from cf8fa9f to 721695dCompare October 17, 2024 20:38

hollymandel force-pushed the issue_7704 branch from c7c71ad to d1ac5beCompare October 17, 2024 21:00