Basic Zarr-python 2.x compatibility changes

[TomAugspurger]

Demonstration of some changes for backwards compatibility being discussed at #2097

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[TomAugspurger]

In 2.x, Group.create was used to create an array in the Group.

I slightly prefer from_* for alternative constructors anyway, so maybe this is a strict improvement and not just a compatibility shim?

[jhamman]

I agree that this needs to change in order to avoid confusion.

[d-v-b]

i'm also a fan of from_X, and from_store seems good

[TomAugspurger]

We can vendor and use https://github.com/scikit-learn/scikit-learn/blob/e87b32a81c70abed8f2e97483758eb64df8255e9/sklearn/utils/validation.py#L32 to deprecate positional arguments. At least of xarray, it passed just store as a positional argument, but deprecating these doesn't feel like too much work.

[jhamman]

I like this suggestion. Should we do it in 2.18.3 or 3.0 though?

[TomAugspurger]

I don't have a strong preference here.

IMO, calendar time is more meaningful than version numbers for giving projects time to adapt to changes. So I'd say deprecate it in 3.0, as long as we don't have a hard stance on requiring keyword only for these APIs in 3.0

[TomAugspurger]

Should we deprecate this, and encourage users to do group.store_path.store?

[d-v-b]

github is flagging this conversation as outdated, but regardless: I agree that we should just have either group.store or group.store_path.store. But I'm not a huge fan of store_path actually, so I'd actually favor deprecating that, and going back to v2 behavior where arrays / groups have a store attribute, and the store has a path attribute. But this is just my personal opinion, built from finding group.store_path.store a little weird.

[TomAugspurger]

Same question as store.

Should probably add this to AsyncGroup too and update this implementation.

[jhamman]

@TomAugspurger - this generally looks like a solid set of fixes. Do you have more here or should try to wrap these up?

[d-v-b]

in principle we could test this functionality for other stores. but I also think testing the full product of [array / group features * stores] is excessive. I guess as long as we are confident that we are densely testing the core store APIs across store types, then it's fine to just test something derived from the store APIs on a single store.

[TomAugspurger]

I think that in this case it's probably OK, since the test here is focused on the behavior of Attributes. This happens to require a reference to a Group (which requires a Store) but hopefully that interface is sufficiently covered by other tests.

[TomAugspurger]

I've confirmed that #2157 fixes the CI failure here.

[jhamman]

Thanks @TomAugspurger for pulling these together. Much improved compatibility with v2.

[TomAugspurger]

Demonstration of some changes for backwards compatibility being discussed at #2097

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[TomAugspurger]

In 2.x, Group.create was used to create an array in the Group.

I slightly prefer from_* for alternative constructors anyway, so maybe this is a strict improvement and not just a compatibility shim?

[jhamman]

I agree that this needs to change in order to avoid confusion.

[d-v-b]

i'm also a fan of from_X, and from_store seems good

[TomAugspurger]

We can vendor and use https://github.com/scikit-learn/scikit-learn/blob/e87b32a81c70abed8f2e97483758eb64df8255e9/sklearn/utils/validation.py#L32 to deprecate positional arguments. At least of xarray, it passed just store as a positional argument, but deprecating these doesn't feel like too much work.

[jhamman]

I like this suggestion. Should we do it in 2.18.3 or 3.0 though?

[TomAugspurger]

I don't have a strong preference here.

IMO, calendar time is more meaningful than version numbers for giving projects time to adapt to changes. So I'd say deprecate it in 3.0, as long as we don't have a hard stance on requiring keyword only for these APIs in 3.0

[TomAugspurger]

Should we deprecate this, and encourage users to do group.store_path.store?

[d-v-b]

github is flagging this conversation as outdated, but regardless: I agree that we should just have either group.store or group.store_path.store. But I'm not a huge fan of store_path actually, so I'd actually favor deprecating that, and going back to v2 behavior where arrays / groups have a store attribute, and the store has a path attribute. But this is just my personal opinion, built from finding group.store_path.store a little weird.

[TomAugspurger]

Same question as store.

Should probably add this to AsyncGroup too and update this implementation.

[jhamman]

@TomAugspurger - this generally looks like a solid set of fixes. Do you have more here or should try to wrap these up?

[d-v-b]

in principle we could test this functionality for other stores. but I also think testing the full product of [array / group features * stores] is excessive. I guess as long as we are confident that we are densely testing the core store APIs across store types, then it's fine to just test something derived from the store APIs on a single store.

[TomAugspurger]

I think that in this case it's probably OK, since the test here is focused on the behavior of Attributes. This happens to require a reference to a Group (which requires a Store) but hopefully that interface is sufficiently covered by other tests.

[TomAugspurger]

I've confirmed that #2157 fixes the CI failure here.

[jhamman]

Thanks @TomAugspurger for pulling these together. Much improved compatibility with v2.

[TomAugspurger]

Demonstration of some changes for backwards compatibility being discussed at #2097

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[TomAugspurger]

In 2.x, Group.create was used to create an array in the Group.

I slightly prefer from_* for alternative constructors anyway, so maybe this is a strict improvement and not just a compatibility shim?

[jhamman]

I agree that this needs to change in order to avoid confusion.

[d-v-b]

i'm also a fan of from_X, and from_store seems good

[TomAugspurger]

We can vendor and use https://github.com/scikit-learn/scikit-learn/blob/e87b32a81c70abed8f2e97483758eb64df8255e9/sklearn/utils/validation.py#L32 to deprecate positional arguments. At least of xarray, it passed just store as a positional argument, but deprecating these doesn't feel like too much work.

[jhamman]

I like this suggestion. Should we do it in 2.18.3 or 3.0 though?

[TomAugspurger]

I don't have a strong preference here.

IMO, calendar time is more meaningful than version numbers for giving projects time to adapt to changes. So I'd say deprecate it in 3.0, as long as we don't have a hard stance on requiring keyword only for these APIs in 3.0

[TomAugspurger]

Should we deprecate this, and encourage users to do group.store_path.store?

[d-v-b]

github is flagging this conversation as outdated, but regardless: I agree that we should just have either group.store or group.store_path.store. But I'm not a huge fan of store_path actually, so I'd actually favor deprecating that, and going back to v2 behavior where arrays / groups have a store attribute, and the store has a path attribute. But this is just my personal opinion, built from finding group.store_path.store a little weird.

[TomAugspurger]

Same question as store.

Should probably add this to AsyncGroup too and update this implementation.

[jhamman]

@TomAugspurger - this generally looks like a solid set of fixes. Do you have more here or should try to wrap these up?

[d-v-b]

in principle we could test this functionality for other stores. but I also think testing the full product of [array / group features * stores] is excessive. I guess as long as we are confident that we are densely testing the core store APIs across store types, then it's fine to just test something derived from the store APIs on a single store.

[TomAugspurger]

I think that in this case it's probably OK, since the test here is focused on the behavior of Attributes. This happens to require a reference to a Group (which requires a Store) but hopefully that interface is sufficiently covered by other tests.

[TomAugspurger]

I've confirmed that #2157 fixes the CI failure here.

[jhamman]

Thanks @TomAugspurger for pulling these together. Much improved compatibility with v2.

[TomAugspurger]

Demonstration of some changes for backwards compatibility being discussed at #2097

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[TomAugspurger]

In 2.x, Group.create was used to create an array in the Group.

I slightly prefer from_* for alternative constructors anyway, so maybe this is a strict improvement and not just a compatibility shim?

[jhamman]

I agree that this needs to change in order to avoid confusion.

[d-v-b]

i'm also a fan of from_X, and from_store seems good

[TomAugspurger]

We can vendor and use https://github.com/scikit-learn/scikit-learn/blob/e87b32a81c70abed8f2e97483758eb64df8255e9/sklearn/utils/validation.py#L32 to deprecate positional arguments. At least of xarray, it passed just store as a positional argument, but deprecating these doesn't feel like too much work.

[jhamman]

I like this suggestion. Should we do it in 2.18.3 or 3.0 though?

[TomAugspurger]

I don't have a strong preference here.

IMO, calendar time is more meaningful than version numbers for giving projects time to adapt to changes. So I'd say deprecate it in 3.0, as long as we don't have a hard stance on requiring keyword only for these APIs in 3.0

[TomAugspurger]

Should we deprecate this, and encourage users to do group.store_path.store?

[d-v-b]

github is flagging this conversation as outdated, but regardless: I agree that we should just have either group.store or group.store_path.store. But I'm not a huge fan of store_path actually, so I'd actually favor deprecating that, and going back to v2 behavior where arrays / groups have a store attribute, and the store has a path attribute. But this is just my personal opinion, built from finding group.store_path.store a little weird.

[TomAugspurger]

Same question as store.

Should probably add this to AsyncGroup too and update this implementation.

[jhamman]

@TomAugspurger - this generally looks like a solid set of fixes. Do you have more here or should try to wrap these up?

[d-v-b]

in principle we could test this functionality for other stores. but I also think testing the full product of [array / group features * stores] is excessive. I guess as long as we are confident that we are densely testing the core store APIs across store types, then it's fine to just test something derived from the store APIs on a single store.

[TomAugspurger]

I think that in this case it's probably OK, since the test here is focused on the behavior of Attributes. This happens to require a reference to a Group (which requires a Store) but hopefully that interface is sufficiently covered by other tests.

[TomAugspurger]

I've confirmed that #2157 fixes the CI failure here.

[jhamman]

Thanks @TomAugspurger for pulling these together. Much improved compatibility with v2.

[TomAugspurger]

Demonstration of some changes for backwards compatibility being discussed at #2097

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[TomAugspurger]

In 2.x, Group.create was used to create an array in the Group.

I slightly prefer from_* for alternative constructors anyway, so maybe this is a strict improvement and not just a compatibility shim?

[jhamman]

I agree that this needs to change in order to avoid confusion.

[d-v-b]

i'm also a fan of from_X, and from_store seems good

[TomAugspurger]

We can vendor and use https://github.com/scikit-learn/scikit-learn/blob/e87b32a81c70abed8f2e97483758eb64df8255e9/sklearn/utils/validation.py#L32 to deprecate positional arguments. At least of xarray, it passed just store as a positional argument, but deprecating these doesn't feel like too much work.

[jhamman]

I like this suggestion. Should we do it in 2.18.3 or 3.0 though?

[TomAugspurger]

I don't have a strong preference here.

IMO, calendar time is more meaningful than version numbers for giving projects time to adapt to changes. So I'd say deprecate it in 3.0, as long as we don't have a hard stance on requiring keyword only for these APIs in 3.0

[TomAugspurger]

Should we deprecate this, and encourage users to do group.store_path.store?

[d-v-b]

github is flagging this conversation as outdated, but regardless: I agree that we should just have either group.store or group.store_path.store. But I'm not a huge fan of store_path actually, so I'd actually favor deprecating that, and going back to v2 behavior where arrays / groups have a store attribute, and the store has a path attribute. But this is just my personal opinion, built from finding group.store_path.store a little weird.

[TomAugspurger]

Same question as store.

Should probably add this to AsyncGroup too and update this implementation.

[jhamman]

@TomAugspurger - this generally looks like a solid set of fixes. Do you have more here or should try to wrap these up?

[d-v-b]

in principle we could test this functionality for other stores. but I also think testing the full product of [array / group features * stores] is excessive. I guess as long as we are confident that we are densely testing the core store APIs across store types, then it's fine to just test something derived from the store APIs on a single store.

[TomAugspurger]

I think that in this case it's probably OK, since the test here is focused on the behavior of Attributes. This happens to require a reference to a Group (which requires a Store) but hopefully that interface is sufficiently covered by other tests.

[TomAugspurger]

I've confirmed that #2157 fixes the CI failure here.

[jhamman]

Thanks @TomAugspurger for pulling these together. Much improved compatibility with v2.

[TomAugspurger]

Demonstration of some changes for backwards compatibility being discussed at #2097

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[TomAugspurger]

In 2.x, Group.create was used to create an array in the Group.

I slightly prefer from_* for alternative constructors anyway, so maybe this is a strict improvement and not just a compatibility shim?

[jhamman]

I agree that this needs to change in order to avoid confusion.

[d-v-b]

i'm also a fan of from_X, and from_store seems good

[TomAugspurger]

We can vendor and use https://github.com/scikit-learn/scikit-learn/blob/e87b32a81c70abed8f2e97483758eb64df8255e9/sklearn/utils/validation.py#L32 to deprecate positional arguments. At least of xarray, it passed just store as a positional argument, but deprecating these doesn't feel like too much work.

[jhamman]

I like this suggestion. Should we do it in 2.18.3 or 3.0 though?

[TomAugspurger]

I don't have a strong preference here.

IMO, calendar time is more meaningful than version numbers for giving projects time to adapt to changes. So I'd say deprecate it in 3.0, as long as we don't have a hard stance on requiring keyword only for these APIs in 3.0

[TomAugspurger]

Should we deprecate this, and encourage users to do group.store_path.store?

[d-v-b]

github is flagging this conversation as outdated, but regardless: I agree that we should just have either group.store or group.store_path.store. But I'm not a huge fan of store_path actually, so I'd actually favor deprecating that, and going back to v2 behavior where arrays / groups have a store attribute, and the store has a path attribute. But this is just my personal opinion, built from finding group.store_path.store a little weird.

[TomAugspurger]

Same question as store.

Should probably add this to AsyncGroup too and update this implementation.

[jhamman]

@TomAugspurger - this generally looks like a solid set of fixes. Do you have more here or should try to wrap these up?

[d-v-b]

in principle we could test this functionality for other stores. but I also think testing the full product of [array / group features * stores] is excessive. I guess as long as we are confident that we are densely testing the core store APIs across store types, then it's fine to just test something derived from the store APIs on a single store.

[TomAugspurger]

I think that in this case it's probably OK, since the test here is focused on the behavior of Attributes. This happens to require a reference to a Group (which requires a Store) but hopefully that interface is sufficiently covered by other tests.

[TomAugspurger]

I've confirmed that #2157 fixes the CI failure here.

[jhamman]

Thanks @TomAugspurger for pulling these together. Much improved compatibility with v2.

[TomAugspurger]

Demonstration of some changes for backwards compatibility being discussed at #2097

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[TomAugspurger]

In 2.x, Group.create was used to create an array in the Group.

I slightly prefer from_* for alternative constructors anyway, so maybe this is a strict improvement and not just a compatibility shim?

[jhamman]

I agree that this needs to change in order to avoid confusion.

[d-v-b]

i'm also a fan of from_X, and from_store seems good

[TomAugspurger]

We can vendor and use https://github.com/scikit-learn/scikit-learn/blob/e87b32a81c70abed8f2e97483758eb64df8255e9/sklearn/utils/validation.py#L32 to deprecate positional arguments. At least of xarray, it passed just store as a positional argument, but deprecating these doesn't feel like too much work.

[jhamman]

I like this suggestion. Should we do it in 2.18.3 or 3.0 though?

[TomAugspurger]

I don't have a strong preference here.

IMO, calendar time is more meaningful than version numbers for giving projects time to adapt to changes. So I'd say deprecate it in 3.0, as long as we don't have a hard stance on requiring keyword only for these APIs in 3.0

[TomAugspurger]

Should we deprecate this, and encourage users to do group.store_path.store?

[d-v-b]

github is flagging this conversation as outdated, but regardless: I agree that we should just have either group.store or group.store_path.store. But I'm not a huge fan of store_path actually, so I'd actually favor deprecating that, and going back to v2 behavior where arrays / groups have a store attribute, and the store has a path attribute. But this is just my personal opinion, built from finding group.store_path.store a little weird.

[TomAugspurger]

Same question as store.

Should probably add this to AsyncGroup too and update this implementation.

[jhamman]

@TomAugspurger - this generally looks like a solid set of fixes. Do you have more here or should try to wrap these up?

[d-v-b]

in principle we could test this functionality for other stores. but I also think testing the full product of [array / group features * stores] is excessive. I guess as long as we are confident that we are densely testing the core store APIs across store types, then it's fine to just test something derived from the store APIs on a single store.

[TomAugspurger]

I think that in this case it's probably OK, since the test here is focused on the behavior of Attributes. This happens to require a reference to a Group (which requires a Store) but hopefully that interface is sufficiently covered by other tests.

[TomAugspurger]

I've confirmed that #2157 fixes the CI failure here.

[jhamman]

Thanks @TomAugspurger for pulling these together. Much improved compatibility with v2.

[TomAugspurger]

Demonstration of some changes for backwards compatibility being discussed at #2097

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[TomAugspurger]

In 2.x, Group.create was used to create an array in the Group.

I slightly prefer from_* for alternative constructors anyway, so maybe this is a strict improvement and not just a compatibility shim?

[jhamman]

I agree that this needs to change in order to avoid confusion.

[d-v-b]

i'm also a fan of from_X, and from_store seems good

[TomAugspurger]

We can vendor and use https://github.com/scikit-learn/scikit-learn/blob/e87b32a81c70abed8f2e97483758eb64df8255e9/sklearn/utils/validation.py#L32 to deprecate positional arguments. At least of xarray, it passed just store as a positional argument, but deprecating these doesn't feel like too much work.

[jhamman]

I like this suggestion. Should we do it in 2.18.3 or 3.0 though?

[TomAugspurger]

I don't have a strong preference here.

IMO, calendar time is more meaningful than version numbers for giving projects time to adapt to changes. So I'd say deprecate it in 3.0, as long as we don't have a hard stance on requiring keyword only for these APIs in 3.0

[TomAugspurger]

Should we deprecate this, and encourage users to do group.store_path.store?

[d-v-b]

github is flagging this conversation as outdated, but regardless: I agree that we should just have either group.store or group.store_path.store. But I'm not a huge fan of store_path actually, so I'd actually favor deprecating that, and going back to v2 behavior where arrays / groups have a store attribute, and the store has a path attribute. But this is just my personal opinion, built from finding group.store_path.store a little weird.

[TomAugspurger]

Same question as store.

Should probably add this to AsyncGroup too and update this implementation.

[jhamman]

@TomAugspurger - this generally looks like a solid set of fixes. Do you have more here or should try to wrap these up?

[d-v-b]

in principle we could test this functionality for other stores. but I also think testing the full product of [array / group features * stores] is excessive. I guess as long as we are confident that we are densely testing the core store APIs across store types, then it's fine to just test something derived from the store APIs on a single store.

[TomAugspurger]

I think that in this case it's probably OK, since the test here is focused on the behavior of Attributes. This happens to require a reference to a Group (which requires a Store) but hopefully that interface is sufficiently covered by other tests.

[TomAugspurger]

I've confirmed that #2157 fixes the CI failure here.

[jhamman]

Thanks @TomAugspurger for pulling these together. Much improved compatibility with v2.