Skip to content

Raise if Index.create_variables returns more variables than passed in through set_xindex - #10503

Merged
dcherian merged 5 commits into
pydata:mainfrom
dhruvak001:issue#10499
Jul 9, 2025
Merged

Raise if Index.create_variables returns more variables than passed in through set_xindex#10503
dcherian merged 5 commits into
pydata:mainfrom
dhruvak001:issue#10499

Conversation

@dhruvak001

@dhruvak001dhruvak001 commented Jul 4, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

Previously, set_xindex silently ignored any extra variables returned by custom indexes' create_variables method, only adding variables that matched the original coordinate names. This prevented custom indexes from providing additional computed variables, limiting their usefulness for advanced indexing scenarios like weather forecasting where derived coordinates are needed.

This change modifies set_xindex in dataset.py to add all variables returned by create_variables(), not just those matching coordinate names. The implementation simply iterates through all returned variables instead of filtering by coordinate names.

The fix enables custom indexes to provide computed variables while maintaining full backward compatibility. Existing code continues to work unchanged, but custom indexes can now return additional variables that will be properly added to the dataset, resolving the silent ignoring behavior that was confusing to users.

@dhruvak001dhruvak001 mentioned this pull request Jul 5, 2025
Closed
@dcherian

dcherian commented Jul 7, 2025

Copy link
Copy Markdown
Contributor

Thanks! given the discussion on the issue, can you change to raise an error if extra variables are returned please? That will be a good improvement until we decide to change behaviour

@dhruvak001

dhruvak001 commented Jul 7, 2025•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

@dcherian I have already implemented it accordingly, as I after then commented (#10499 (comment)) on issue too. I think you can review it.

@dcheriandcherian changed the title Allow custom indexes to create new variablesRaise if create_variables returns more variables than passed in through set_xindexJul 8, 2025
@dcheriandcherian changed the title Raise if create_variables returns more variables than passed in through set_xindexRaise if Index.create_variables returns more variables than passed in through set_xindexJul 8, 2025
@dcherian

dcherian commented Jul 8, 2025

Copy link
Copy Markdown
Contributor

Nice, can you add a test to https://github.com/pydata/xarray/blob/main/xarray/tests/test_indexes.py please

@dhruvak001

dhruvak001 commented Jul 8, 2025

Copy link
Copy Markdown
ContributorAuthor

@dcherian i have added the tests.

dcherian
dcherian reviewed Jul 8, 2025
View reviewed changes
Comment threadxarray/tests/test_indexes.py Outdated
dcherian
dcherian reviewed Jul 8, 2025
View reviewed changes
Comment threadxarray/tests/test_indexes.py Outdated
dcherian
dcherian reviewed Jul 8, 2025
View reviewed changes
Comment threadxarray/tests/test_indexes.py Outdated
dcherian
dcherian reviewed Jul 8, 2025
View reviewed changes
Comment threadxarray/tests/test_indexes.py Outdated
dcherian
dcherian reviewed Jul 8, 2025
View reviewed changes
Comment threadxarray/tests/test_indexes.py Outdated
dcherian
dcherian reviewed Jul 8, 2025
View reviewed changes
Comment threadxarray/tests/test_indexes.py Outdated
@dhruvak001
dhruvak001 requested a review from dcherianJuly 9, 2025 10:38
dcherian
dcherian approved these changes Jul 9, 2025
View reviewed changes

@dcheriandcherian left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing, thanks!

dcherian
dcherian reviewed Jul 9, 2025
View reviewed changes
Comment threadxarray/tests/test_indexes.py
@dcheriandcherian added the plan to merge Final call for comments label Jul 9, 2025
@dcherian
dcherian merged commit d85185b into pydata:mainJul 9, 2025
@benbovybenbovy mentioned this pull request Jul 9, 2025
Merged
2 tasks
@dhruvak001
dhruvak001 deleted the issue#10499 branch July 9, 2025 17:06
@benbovybenbovy mentioned this pull request Jul 9, 2025
Closed
dcherian added a commit to jsignell/xarray that referenced this pull request Jul 18, 2025
@dcherian
Merge branch 'main' into concat_default_kwargs
518cd6e
* main: (94 commits)
Restructure API documentation into organized sections (pydata#10540)
Fix convert calendar with different dimension name (pydata#10544)
Fix kerchunk error in docs build (pydata#10545)
Call super().__init__() in st.SearchStrategy subclasses (pydata#10543)
CI: update actions location (pydata#10529)
meta: Fix labeler topic-documentation (pydata#10524)
drop_vars: use emit_user_level_warning (pydata#10528)
Pydap flaky tests (pydata#10525)
Remove test skip (pydata#10523)
Improve warning message and tests for timedelta decoding (pydata#10508)
Add dev whats-new (pydata#10522)
Add release notes for v2025.07.1 (pydata#10520)
Updates for Zarr 3 Dtypes (pydata#10456)
Update Custom Indexes section in api.rst (pydata#10517)
Expression not assigned (pydata#10507)
Raise if `Index.create_variables` returns more variables than passed in through `set_xindex` (pydata#10503)
Allow setting (or skipping) new indexes in open_dataset (pydata#8051)
doc: add Pandas(Multi)Index to api reference (pydata#10515)
Update pre-commit hooks (pydata#10510)
Update pre-commit ruff legacy alias (pydata#10511)
...
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dcheriandcheriandcherian approved these changes

Assignees

No one assigned

Labels

plan to mergeFinal call for commentstopic-documentation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Allow custom indexes to create new variables

2 participants

@dhruvak001@dcherian