Hypothesis strategy for generating Variable objects by TomNicholas · Pull Request #8404 · pydata/xarray

TomNicholas · 2023-11-02T17:04:03Z

Breaks out just the part of #6908 needed for generating arbitrary xarray.Variable objects. (so ignore the ginormous number of commits)

EDIT: Check out this test which performs a mean on any subset of any Variable object!

In [36]: fromxarray.testing.strategiesimportvariablesIn [37]: variables().example()
<xarray.Variable (ĭ: 3)>array([-2.22507386e-313-6.62447795e+016j,
nan-6.46207519e+185j,
-2.22507386e-309+3.33333333e-001j])

@andersy005 @maxrjones @jhamman I thought this might be useful for the NamedArray testing. (xref #8370 and #8244)

@keewis and @Zac-HD sorry for letting that PR languish for literally a year 😅 This PR addresses your feedback about accepting a callable that returns a strategy generating arrays. That suggestion makes some things a bit more complex in user code but actually allows me to simplify the internals of the variables strategy significantly. I'm actually really happy with this PR - I think it solves what we were discussing, and is a sensible checkpoint to merge before going back to making strategies for generating composite objects like DataArrays/Datasets work.

Closes part of Public hypothesis strategies for generating xarray data #6911
Tests added
User visible changes (including notable bug fixes) are documented in whats-new.rst
New functions/methods are listed in api.rst

…ctor

for more information, see https://pre-commit.ci

…atible

…s/xarray into hypothesis-strategies

for more information, see https://pre-commit.ci

keewis

I didn't have time to check the tests yet, but here are a few comments

keewis · 2023-11-13T17:48:29Z

+Testing your code
+=================


Not sure. It is true that the page has a different target audience than the other pages in the user guide, but then again applications can also be tested. And, so far the "internals" section describes implementation details or extension mechanisms that affect the internals.

keewis · 2023-11-13T18:18:49Z

+ return (
+ npst.integer_dtypes()
+ | npst.unsigned_integer_dtypes()
+ | npst.floating_dtypes()
+ | npst.complex_number_dtypes()
+ )


we do support string dtypes, but only for a subset of operations. Is this worth mentioning?

This is not meant to be an exhaustive list (yet). It doesn't include datetimes either.

agreed, but most operations don't make sense on string or datetime dtypes so it might be better to make a separate list of dtypes for those?

Sure - I'm just saying let's defer detailed discussions of which types to test until another issue / PR, the point of this PR is to provide a framework flexible enough to easily test xarray functions with any type we want, which this achieves.

Co-authored-by: Justus Magin <keewis@users.noreply.github.com>

…omNicholas/xarray into hypothesis-strategies-variable

Zac-HD · 2023-11-13T22:44:21Z

The only final thing is that the docs don't build because of one weird warning (our docs are set to fail on any warnings):
xarray/xarray/testing/strategies.py:docstring of xarray.testing.strategies.accept.<locals>.variables:47:
WARNING: Block quote ends without a blank line; unexpected unindent.
Given that I don't define any local variables called accept, but hypothesis apparently does, I guess this must be hypothesis' fault somehow?

My guess is that this is an existing docstring, the location of which is being misreported due to the various wrappers that Hypothesis inserts. I'd be very surprised if Hypothesis is modifying docstrings somehow, but I guess trimming trailing whitespace is the kind of thing that could happen somewhere in the stack.

No direct insight, but getting the full text of the docstring it's complaining about should help?

…ategy

TomNicholas · 2023-12-05T05:53:27Z

I got the docs build to pass! The warning was due to extra lines in the examples of the variables strategy docstring. I only managed to find it by trial and error 🙄

@keewis do you want to review the tests before I merge it? (The test failures now are something groupby-related, and are also happening in #8521, so definitely not my fault!)

keewis

I didn't spot anything that we wouldn't be able to change after merging / releasing, so I'd say let's merge and see how well it works in practice.

keewis · 2023-12-05T19:52:19Z

+ )
+
+
+def smallish_arrays(


is the only reason we have this function the default strategy for shape (and maybe some additional typing)? If so, we might be able to use functools.partial on npst.arrays? Unless you meant to expose this as public API (it's not in the API reference)

That is the only reason. I did not think of using functools.partial - that's a good idea, I can try that out before merging.

That actually won't work because we do need to be able to pass shape and dtype to the array_strategy_fn.
But I tried removing smallish_arrays completely and the tests still seem to complete in a reasonable amount of time, so I've actually just taken it out for now.

…ray#8404

* fix import of xarray.testing internals that was changed by pydata/xarray#8404 * bump minimum required version of xarray * linting

* main: (26 commits) Filter null values before plotting (pydata#8535) Update concat.py (pydata#8538) Add getitem to array protocol (pydata#8406) Added option to specify weights in xr.corr() and xr.cov() (pydata#8527) Filter out doctest warning (pydata#8539) Bump actions/setup-python from 4 to 5 (pydata#8540) Point users to where in their code they should make mods for Dataset.dims (pydata#8534) Add Cumulative aggregation (pydata#8512) dev whats-new Whats-new for 2023.12.0 (pydata#8532) explicitly skip using `__array_namespace__` for `numpy.ndarray` (pydata#8526) Add `eval` method to Dataset (pydata#7163) Deprecate ds.dims returning dict (pydata#8500) test and fix empty xindexes repr (pydata#8521) Remove PR labeler bot (pydata#8525) Hypothesis strategy for generating Variable objects (pydata#8404) Use numbagg for `rolling` methods (pydata#8493) Bump pypa/gh-action-pypi-publish from 1.8.10 to 1.8.11 (pydata#8514) fix RTD docs build (pydata#8519) Fix type of `.assign_coords` (pydata#8495) ...

* main: (58 commits) Adapt map_blocks to use new Coordinates API (pydata#8560) add xeofs to ecosystem.rst (pydata#8561) Offer a fixture for unifying DataArray & Dataset tests (pydata#8533) Generalize cumulative reduction (scan) to non-dask types (pydata#8019) Filter null values before plotting (pydata#8535) Update concat.py (pydata#8538) Add getitem to array protocol (pydata#8406) Added option to specify weights in xr.corr() and xr.cov() (pydata#8527) Filter out doctest warning (pydata#8539) Bump actions/setup-python from 4 to 5 (pydata#8540) Point users to where in their code they should make mods for Dataset.dims (pydata#8534) Add Cumulative aggregation (pydata#8512) dev whats-new Whats-new for 2023.12.0 (pydata#8532) explicitly skip using `__array_namespace__` for `numpy.ndarray` (pydata#8526) Add `eval` method to Dataset (pydata#7163) Deprecate ds.dims returning dict (pydata#8500) test and fix empty xindexes repr (pydata#8521) Remove PR labeler bot (pydata#8525) Hypothesis strategy for generating Variable objects (pydata#8404) ...

commit 0a0f800 Merge: 33c803341d33f5 Author: Deepak Cherian <dcherian@users.noreply.github.com> Date: Tue Jan 2 20:42:51 2024 -0700 Merge branch 'main' into depr-groupby-squeeze-2 commit 33c8033 Author: Deepak Cherian <deepak@cherian.net> Date: Tue Jan 2 20:40:42 2024 -0700 Don't skip for resampling commit d7be352 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed Jan 3 03:24:13 2024 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit d13fa0e Author: Deepak Cherian <dcherian@users.noreply.github.com> Date: Tue Jan 2 20:23:43 2024 -0700 Apply suggestions from code review Co-authored-by: Michael Niklas <mick.niklas@gmail.com> commit dd6ea53 Author: Deepak Cherian <deepak@cherian.net> Date: Thu Dec 21 19:29:40 2023 -0700 Silence more warnings commit 44e5a41 Author: Deepak Cherian <deepak@cherian.net> Date: Thu Dec 21 19:21:06 2023 -0700 minimize test mods commit 94c1c1f Author: Deepak Cherian <deepak@cherian.net> Date: Thu Dec 21 18:55:46 2023 -0700 Add tests for pydata#8263 commit 0ab4eb6 Author: Deepak Cherian <deepak@cherian.net> Date: Thu Dec 21 18:47:41 2023 -0700 Fix typing commit a064430 Merge: d6a3f2d03ec3cb Author: Deepak Cherian <deepak@cherian.net> Date: Thu Dec 21 18:47:04 2023 -0700 Merge branch 'main' into depr-groupby-squeeze-2 * main: Fix mypy type ignore (pydata#8564) Support for the new compression arguments. (pydata#7551) FIX: reverse index output of bottleneck move_argmax/move_argmin functions (pydata#8552) Adapt map_blocks to use new Coordinates API (pydata#8560) add xeofs to ecosystem.rst (pydata#8561) Offer a fixture for unifying DataArray & Dataset tests (pydata#8533) Generalize cumulative reduction (scan) to non-dask types (pydata#8019) commit d6a3f2d Author: Deepak Cherian <deepak@cherian.net> Date: Thu Dec 21 18:46:50 2023 -0700 Fix generator for aggregations commit 97f1695 Author: Deepak Cherian <deepak@cherian.net> Date: Tue Dec 19 10:58:11 2023 -0700 Fix docs commit 5b33b98 Author: Deepak Cherian <deepak@cherian.net> Date: Sun Dec 17 20:35:53 2023 -0700 fix whats-new commit 80b2b36 Author: Deepak Cherian <deepak@cherian.net> Date: Sun Dec 17 20:26:17 2023 -0700 Reduce more warnings commit 5f6f4ea Merge: a57d4ae2971994 Author: Deepak Cherian <deepak@cherian.net> Date: Sat Dec 16 20:33:13 2023 -0700 Merge branch 'main' into depr-groupby-squeeze-2 * main: (26 commits) Filter null values before plotting (pydata#8535) Update concat.py (pydata#8538) Add getitem to array protocol (pydata#8406) Added option to specify weights in xr.corr() and xr.cov() (pydata#8527) Filter out doctest warning (pydata#8539) Bump actions/setup-python from 4 to 5 (pydata#8540) Point users to where in their code they should make mods for Dataset.dims (pydata#8534) Add Cumulative aggregation (pydata#8512) dev whats-new Whats-new for 2023.12.0 (pydata#8532) explicitly skip using `__array_namespace__` for `numpy.ndarray` (pydata#8526) Add `eval` method to Dataset (pydata#7163) Deprecate ds.dims returning dict (pydata#8500) test and fix empty xindexes repr (pydata#8521) Remove PR labeler bot (pydata#8525) Hypothesis strategy for generating Variable objects (pydata#8404) Use numbagg for `rolling` methods (pydata#8493) Bump pypa/gh-action-pypi-publish from 1.8.10 to 1.8.11 (pydata#8514) fix RTD docs build (pydata#8519) Fix type of `.assign_coords` (pydata#8495) ... commit a57d4ae Author: Deepak Cherian <deepak@cherian.net> Date: Fri Dec 1 21:36:04 2023 -0700 Test one more warning commit bf8139d Author: Deepak Cherian <dcherian@users.noreply.github.com> Date: Fri Dec 1 21:33:45 2023 -0700 Update xarray/tests/test_groupby.py commit 4e9a063 Author: Deepak Cherian <deepak@cherian.net> Date: Fri Dec 1 21:10:14 2023 -0700 Set squeeze=None for Dataset too commit c2e576e Author: Deepak Cherian <deepak@cherian.net> Date: Fri Dec 1 20:54:17 2023 -0700 Fix first, last commit 6d8e822 Author: Deepak Cherian <deepak@cherian.net> Date: Fri Dec 1 20:46:21 2023 -0700 better warning commit 62c334b Author: Deepak Cherian <deepak@cherian.net> Date: Fri Dec 1 20:45:17 2023 -0700 silence warnings commit b7805a8 Author: dcherian <deepak@cherian.net> Date: Tue Aug 15 10:54:25 2023 -0600 Deprecate `squeeze` in GroupBy. Closespydata#2157

* fix import of xarray.testing internals that was changed by pydata/xarray#8404 * bump minimum required version of xarray * linting

TomNicholasand others added 30 commits August 11, 2022 03:11

copied files defining strategies over to this branch
587ebb8

placed testing functions in their own directory
acbfa69

moved hypothesis strategies into new testing directory
73d763f

begin type hinting strategies
db2deff

renamed strategies for consistency with hypothesis conventions
746cfc8

added strategies to public API (with experimental warning)
03cd9de

strategies for chunking patterns
2fe3583

rewrote variables strategy to have same signature as Variable constru…
4db3629
…ctor

test variables strategy
14d11aa

fixed most tests
418a359

added helpers so far to API docs
c8a7d0e

add hypothesis to docs CI env
d48aceb

add todo about attrs
a20e341

draft of new user guide page on testing
3a4816f

types for dataarrays strategy
d0406a2

draft for chained chunking example
65a222d

[pre-commit.ci] auto fixes from pre-commit.com hooks
e1d718a
for more information, see https://pre-commit.ci

only accept strategy objects
57d0f5b

fixed failure with passing in two custom strategies that must be comp…
82c734c
…atible

syntax error in example
029f19a

allow sizes dict as argument to variables
46895fe

copied subsequences_of strategy
50c62e9

coordinate_variables generates non-dimensional coords
e21555a

dataarrays strategy given nothing working!
1688779

improved docstrings
0a29d32

datasets strategy works (given nothing)
3259849

Merge branch 'hypothesis-strategies' ofhttps://github.com/TomNichola…
717fabe
…s/xarray into hypothesis-strategies

[pre-commit.ci] auto fixes from pre-commit.com hooks
d76e5b6
for more information, see https://pre-commit.ci

pass dims or data to dataarrays() strategy
c25940c

importorskip hypothesis in tests
cd7b065

[pre-commit.ci] auto fixes from pre-commit.com hooks
afd526d
for more information, see https://pre-commit.ci

keewis reviewed Nov 13, 2023
View reviewed changes

TomNicholasand others added 7 commits November 13, 2023 13:19

Use .copy in convert_to_sparse
6bbd13b
Co-authored-by: Justus Magin <keewis@users.noreply.github.com>

Use st.builds in sparse example
29ecd7d
Co-authored-by: Justus Magin <keewis@users.noreply.github.com>

correct intersphinx link in whatsnew
631e810

Merge branch 'hypothesis-strategies-variable' ofhttps://github.com/T…
c613027
…omNicholas/xarray into hypothesis-strategies-variable

rename module containing assertion functions
4412d98

clarify sentence
1ea0dcf

add general ImportError if hypothesis not installed
cf1a45e

TomNicholas added 3 commits November 14, 2023 10:20

add See Also link to strategies docs page from docstring of every str…
ea738cd
…ategy

typo in ImportError message
79b0094

Merge branch 'main' into hypothesis-strategies-variable
c6d43ca

TomNicholas mentioned this pull request Nov 29, 2023
Repeated coordinates leads to unintuitive (broken?) indexing behaviour #3731
Open

TomNicholas added 2 commits December 4, 2023 16:21

Merge branch 'main' into hypothesis-strategies-variable
00079bd

remove extra blank lines in examples
cbcd486

keewis approved these changes Dec 5, 2023
View reviewed changes

TomNicholas added 2 commits December 5, 2023 17:10

remove smallish_arrays
69ddd08

Merge branch 'main' into hypothesis-strategies-variable
ea90162

TomNicholas merged commit ab6a255 into pydata:mainDec 5, 2023

TomNicholas added a commit to TomNicholas/datatree that referenced this pull request Dec 10, 2023
fix import of xarray.testing internals that was changed bypydata/xar…
eb19bd5
…ray#8404

TomNicholas added a commit to xarray-contrib/datatree that referenced this pull request Dec 10, 2023
Fix for xarray v2023.12.0 (#294)
d3b2a6d
* fix import of xarray.testing internals that was changed by pydata/xarray#8404 * bump minimum required version of xarray * linting

TomNicholas mentioned this pull request Dec 14, 2023
Duckarray tests for constructors and properties #6903
Open
4 tasks

TomNicholas mentioned this pull request Dec 18, 2023
Support non-str Hashables in DataArray #8559
Merged
3 tasks

Zac-HD mentioned this pull request Apr 1, 2024
Hypothesis strategies in xarray.testing.strategies #6908
Open
4 tasks

		Testing your code
		=================

		)


		def smallish_arrays(

Uh oh!

Uh oh!

Conversation

TomNicholas commented Nov 2, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

keewis left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Zac-HD commented Nov 13, 2023

Uh oh!

TomNicholas commented Dec 5, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

keewis left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

TomNicholas commented Nov 2, 2023•
edited
Loading

TomNicholas commented Dec 5, 2023•
edited
Loading