Per-variable specification of boolean parameters in open_dataset by Ostheer · Pull Request #9218 · pydata/xarray

Ostheer · 2024-07-08T13:42:17Z

Added feature to choose whether to apply mask_and_scale for each DataArray in a Dataset individually.

The mask_and_scale parameter can now not only be bool, but also a mapping from variable name (str) to bool.

- [?] User visible changes (including notable bug fixes) are documented in whats-new.rst

New functions/methods are listed in api.rst

I wasn't sure if I was supposed to edit whats-new.rst, so I didn't touch it.

for more information, see https://pre-commit.ci

Ostheer · 2024-07-12T12:25:39Z

Can someone tell me if and how I should edit the what's new file?

I could add something like

- Allow per-variable specification of `mask_and_scale` in `open_dataset` (:pull:`9218`).
By `Mathijs Verhaegh <https://github.com/Ostheer>`_.

But I'm not sure where, as I don't know into which upcoming release this commit would be included.

max-sixty · 2024-07-12T16:25:53Z

This looks good!

Great re whatsnew suggestion, within New Features works!

headtr1ck · 2024-07-12T22:39:03Z

Could also easily extend to other arguments, like decode_times. What do you think?

Co-authored-by: Michael Niklas <mick.niklas@gmail.com>

Otherwise you lose all typing when you use that because it returns Any.

for more information, see https://pre-commit.ci

Ostheer · 2024-07-12T23:09:04Z

Could also easily extend to other arguments, like decode_times. What do you think?

Added for 4 more params (46de50f)

Feature now affects:

mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters

headtr1ck

Add an entry in what's new and this is good! Thanks

for more information, see https://pre-commit.ci

…nto granular-mask-and-scale

for more information, see https://pre-commit.ci

Ostheer · 2024-07-13T11:38:48Z

I resolved some minor Mypy issues stemming from the annotations on _item_or_default, but the Mypy and Mypy 3.9 tasks are still failing.

I can't find how the errors in the logs relate to any of the code changes. Any ideas?

headtr1ck · 2024-07-13T12:24:19Z

I resolved some minor Mypy issues stemming from the annotations on _item_or_default, but the Mypy and Mypy 3.9 tasks are still failing.
I can't find how the errors in the logs relate to any of the code changes. Any ideas?

These checks are failing on main as well. There are already some PRs open to fix these.

I would say this looks good.

Ostheer · 2024-07-13T12:40:38Z

I resolved some minor Mypy issues stemming from the annotations on _item_or_default, but the Mypy and Mypy 3.9 tasks are still failing.
I can't find how the errors in the logs relate to any of the code changes. Any ideas?
These checks are failing on main as well. There are already some PRs open to fix these.
I would say this looks good.

I see. Can we then somehow merge this despite the failing checks? I don't think Github lets me.

headtr1ck · 2024-07-13T13:40:44Z

Yes, no worries.
Let's keep it a day or two for other people to check and then merge. Thanks!

max-sixty · 2024-07-16T06:25:36Z

Thanks @Ostheer !

welcome · 2024-07-16T06:25:43Z

Congratulations on completing your first pull request! Welcome to Xarray! We are proud of you, and hope to see you again!

* main: Enable pandas type checking (pydata#9213) Per-variable specification of boolean parameters in open_dataset (pydata#9218) test push Added a space to the documentation (pydata#9247) Fix typing for test_plot.py (pydata#9234) Allow mypy to run in vscode (pydata#9239) Revert "Test main push" Test main push Revert "Update _typing.py" Update _typing.py Add a `.drop_attrs` method (pydata#8258)

* main: add backend intro and how-to diagram (#9175) Fix copybutton for multi line examples in double digit ipython cells (#9264) Update signature for _arrayfunction.__array__ (#9237) Add encode_cf_datetime benchmark (#9262) groupby, resample: Deprecate some positional args (#9236) Delete ``base`` and ``loffset`` parameters to resample (#9233) Update dropna docstring (#9257) Grouper, Resampler as public api (#8840) Fix mypy on main (#9252) fix fallback isdtype method (#9250) Enable pandas type checking (#9213) Per-variable specification of boolean parameters in open_dataset (#9218) test push Added a space to the documentation (#9247) Fix typing for test_plot.py (#9234)

* main: (54 commits) Adding `open_datatree` backend-specific keyword arguments (#9199) [pre-commit.ci] pre-commit autoupdate (#9202) Restore ability to specify _FillValue as Python native integers (#9258) add backend intro and how-to diagram (#9175) Fix copybutton for multi line examples in double digit ipython cells (#9264) Update signature for _arrayfunction.__array__ (#9237) Add encode_cf_datetime benchmark (#9262) groupby, resample: Deprecate some positional args (#9236) Delete ``base`` and ``loffset`` parameters to resample (#9233) Update dropna docstring (#9257) Grouper, Resampler as public api (#8840) Fix mypy on main (#9252) fix fallback isdtype method (#9250) Enable pandas type checking (#9213) Per-variable specification of boolean parameters in open_dataset (#9218) test push Added a space to the documentation (#9247) Fix typing for test_plot.py (#9234) Allow mypy to run in vscode (#9239) Revert "Test main push" ...

…monotonic-variable * main: (995 commits) Adding `open_datatree` backend-specific keyword arguments (pydata#9199) [pre-commit.ci] pre-commit autoupdate (pydata#9202) Restore ability to specify _FillValue as Python native integers (pydata#9258) add backend intro and how-to diagram (pydata#9175) Fix copybutton for multi line examples in double digit ipython cells (pydata#9264) Update signature for _arrayfunction.__array__ (pydata#9237) Add encode_cf_datetime benchmark (pydata#9262) groupby, resample: Deprecate some positional args (pydata#9236) Delete ``base`` and ``loffset`` parameters to resample (pydata#9233) Update dropna docstring (pydata#9257) Grouper, Resampler as public api (pydata#8840) Fix mypy on main (pydata#9252) fix fallback isdtype method (pydata#9250) Enable pandas type checking (pydata#9213) Per-variable specification of boolean parameters in open_dataset (pydata#9218) test push Added a space to the documentation (pydata#9247) Fix typing for test_plot.py (pydata#9234) Allow mypy to run in vscode (pydata#9239) Revert "Test main push" ...

* main: (54 commits) Adding `open_datatree` backend-specific keyword arguments (pydata#9199) [pre-commit.ci] pre-commit autoupdate (pydata#9202) Restore ability to specify _FillValue as Python native integers (pydata#9258) add backend intro and how-to diagram (pydata#9175) Fix copybutton for multi line examples in double digit ipython cells (pydata#9264) Update signature for _arrayfunction.__array__ (pydata#9237) Add encode_cf_datetime benchmark (pydata#9262) groupby, resample: Deprecate some positional args (pydata#9236) Delete ``base`` and ``loffset`` parameters to resample (pydata#9233) Update dropna docstring (pydata#9257) Grouper, Resampler as public api (pydata#8840) Fix mypy on main (pydata#9252) fix fallback isdtype method (pydata#9250) Enable pandas type checking (pydata#9213) Per-variable specification of boolean parameters in open_dataset (pydata#9218) test push Added a space to the documentation (pydata#9247) Fix typing for test_plot.py (pydata#9234) Allow mypy to run in vscode (pydata#9239) Revert "Test main push" ...

Mathijs Verhaeghand others added 4 commits July 8, 2024 15:24

allow per-variable choice of mask_and_scale in open_dataset
e63d15c

simplify docstring datatype
4bbb924

[pre-commit.ci] auto fixes from pre-commit.com hooks
fa90e7f
for more information, see https://pre-commit.ci

Merge branch 'pydata:main' into granular-mask-and-scale
b70f00f

max-sixty added the plan to merge Final call for comments label Jul 12, 2024

headtr1ck reviewed Jul 12, 2024
View reviewed changes

Comment threadxarray/backends/api.py Outdated
Comment threadxarray/conventions.py Outdated

Ostheerand others added 4 commits July 13, 2024 00:39

dict -> Mapping in type annotation
b218b3c
Co-authored-by: Michael Niklas <mick.niklas@gmail.com>

use typevar for _item_or_default annotation
72483cb
Otherwise you lose all typing when you use that because it returns Any.

[pre-commit.ci] auto fixes from pre-commit.com hooks
a47363c
for more information, see https://pre-commit.ci

implement feature for 4 additional parameters
46de50f

Ostheer changed the title ~~Granular mask and scale~~Per-variable specification of boolean parameters in open_datasetJul 12, 2024

fix default value inconsistency
28e01ab

headtr1ck approved these changes Jul 13, 2024
View reviewed changes

Comment threadxarray/conventions.py Outdated

Mathijs Verhaeghand others added 7 commits July 13, 2024 12:05

add what's new + None annotation
a194d0d

Merge branch 'main' into granular-mask-and-scale
83dfcc1

[pre-commit.ci] auto fixes from pre-commit.com hooks
0f1e432
for more information, see https://pre-commit.ci

_item_or_default return type T | None
5eecad1

Merge branch 'granular-mask-and-scale' of github.com:Ostheer/xarray i…
11a60e3
…nto granular-mask-and-scale

remove deault default value _item_or_default
bc664f6

[pre-commit.ci] auto fixes from pre-commit.com hooks
9e02728
for more information, see https://pre-commit.ci

headtr1ck added the topic-backends label Jul 13, 2024

headtr1ck reviewed Jul 15, 2024
View reviewed changes

Comment threadxarray/backends/api.py Outdated

docstring dtype naming
5762046

max-sixty merged commit 7477fd1 into pydata:mainJul 16, 2024

Uh oh!

Uh oh!

Conversation

Ostheer commented Jul 8, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Ostheer commented Jul 12, 2024

Uh oh!

max-sixty commented Jul 12, 2024

Uh oh!

Uh oh!

Uh oh!

headtr1ck commented Jul 12, 2024

Uh oh!

Ostheer commented Jul 12, 2024

Uh oh!

headtr1ck left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Ostheer commented Jul 13, 2024

Uh oh!

headtr1ck commented Jul 13, 2024

Uh oh!

Ostheer commented Jul 13, 2024

Uh oh!

headtr1ck commented Jul 13, 2024

Uh oh!

Uh oh!

max-sixty commented Jul 16, 2024

Uh oh!

welcomeBot commented Jul 16, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Ostheer commented Jul 8, 2024•
edited
Loading