Feature: Top level V3 API

[jhamman]

Marking as Work In Progress for now. Currently waiting on #1670 to merge.

Closes#1598
Fixes#1019

Open questions I could use feedback on:

1. I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?
2. I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

More soon.

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

cc @d-v-b, @aldenks

[jhamman]

This is ready for an initial review. To set expectations, I don't plan to address the following optional arguments in this PR:

• chunk_store
• cache_attrs
• synchronizer
• meta_array
• storage_options
• dimension_separator
• write_empty_chunks

These will need to be handled in later PRs.

[normanrz]

Suggested change

	MEMORY_ORDER=Literal["C", "F"]
	MemoryOrder=Literal["C", "F"]

[normanrz]

Can we use ChunkCoords here?

[jhamman]

yes, but maybe not for long. I went ahead and changed it but ShapeLike was intended to be int | tuple[int, ...]... Perhaps that will work for ChunkCoords too?

[d-v-b]

instead of mutating kwargs, can we copy it, and make changes to the copy, and return it?

[jhamman]

Good call. This function was coped from v2 but is now refactored along the lines of what you were suggesting.

[d-v-b]

I added a few minor comments, none of which are blockers. A few high-level thoughts:

A lot of the routines zarr.convenience seem strange to me, e.g. save has a very weird function signature that we should probably give a second thought when the dust settles from the v3 release. But changes like that are out of scope for this work.

I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?

This works for me.

I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

Definitely prefer what we have here over async_ones. I don't see any drawbacks to this organization, since presumably we want everything to rest on the async foundation, but I don't have much experience with libraries that expose async and sync apis.

[will-moore]

Hi, thanks for all the work on this.
I'm looking at using this from https://github.com/ome/ome-zarr-py and seeing this error when accessing via Dask:

 File "/Users/wmoore/Desktop/ZARR/ome-zarr-py/ome_zarr/io.py", line 177, in load
return da.from_zarr(self.__store, subpath)
File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/dask/array/core.py", line 3597, in from_zarr
z = zarr.Array(url, read_only=True, path=component, **kwargs)
TypeError: Array.__init__() got an unexpected keyword argument 'read_only'

Is read_only another argument to add to the list above at #1884 (comment) ?

[jhamman]

Hi @will-moore -- interesting to find out that Dask is using the array constructor directly. I would have expected to see zarr.array(...). This PR is covering the API methods formerly found in zarr.convenience and zarr.creation. We'll have to do a bit of work to handle the case you've shown here.

btw, zarr.array(url, read_only_true, path=component) should work with this PR.

[jhamman]

I'm planning to merge this in the morning unless I get further reviews.

[jhamman]

Marking as Work In Progress for now. Currently waiting on #1670 to merge.

Closes#1598
Fixes#1019

Open questions I could use feedback on:

1. I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?
2. I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

More soon.

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

cc @d-v-b, @aldenks

[jhamman]

This is ready for an initial review. To set expectations, I don't plan to address the following optional arguments in this PR:

• chunk_store
• cache_attrs
• synchronizer
• meta_array
• storage_options
• dimension_separator
• write_empty_chunks

These will need to be handled in later PRs.

[normanrz]

Suggested change

	MEMORY_ORDER=Literal["C", "F"]
	MemoryOrder=Literal["C", "F"]

[normanrz]

Can we use ChunkCoords here?

[jhamman]

yes, but maybe not for long. I went ahead and changed it but ShapeLike was intended to be int | tuple[int, ...]... Perhaps that will work for ChunkCoords too?

[d-v-b]

instead of mutating kwargs, can we copy it, and make changes to the copy, and return it?

[jhamman]

Good call. This function was coped from v2 but is now refactored along the lines of what you were suggesting.

[d-v-b]

I added a few minor comments, none of which are blockers. A few high-level thoughts:

A lot of the routines zarr.convenience seem strange to me, e.g. save has a very weird function signature that we should probably give a second thought when the dust settles from the v3 release. But changes like that are out of scope for this work.

I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?

This works for me.

I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

Definitely prefer what we have here over async_ones. I don't see any drawbacks to this organization, since presumably we want everything to rest on the async foundation, but I don't have much experience with libraries that expose async and sync apis.

[will-moore]

Hi, thanks for all the work on this.
I'm looking at using this from https://github.com/ome/ome-zarr-py and seeing this error when accessing via Dask:

 File "/Users/wmoore/Desktop/ZARR/ome-zarr-py/ome_zarr/io.py", line 177, in load
return da.from_zarr(self.__store, subpath)
File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/dask/array/core.py", line 3597, in from_zarr
z = zarr.Array(url, read_only=True, path=component, **kwargs)
TypeError: Array.__init__() got an unexpected keyword argument 'read_only'

Is read_only another argument to add to the list above at #1884 (comment) ?

[jhamman]

Hi @will-moore -- interesting to find out that Dask is using the array constructor directly. I would have expected to see zarr.array(...). This PR is covering the API methods formerly found in zarr.convenience and zarr.creation. We'll have to do a bit of work to handle the case you've shown here.

btw, zarr.array(url, read_only_true, path=component) should work with this PR.

[jhamman]

I'm planning to merge this in the morning unless I get further reviews.

[jhamman]

Marking as Work In Progress for now. Currently waiting on #1670 to merge.

Closes#1598
Fixes#1019

Open questions I could use feedback on:

1. I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?
2. I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

More soon.

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

cc @d-v-b, @aldenks

[jhamman]

This is ready for an initial review. To set expectations, I don't plan to address the following optional arguments in this PR:

• chunk_store
• cache_attrs
• synchronizer
• meta_array
• storage_options
• dimension_separator
• write_empty_chunks

These will need to be handled in later PRs.

[normanrz]

Suggested change

	MEMORY_ORDER=Literal["C", "F"]
	MemoryOrder=Literal["C", "F"]

[normanrz]

Can we use ChunkCoords here?

[jhamman]

yes, but maybe not for long. I went ahead and changed it but ShapeLike was intended to be int | tuple[int, ...]... Perhaps that will work for ChunkCoords too?

[d-v-b]

instead of mutating kwargs, can we copy it, and make changes to the copy, and return it?

[jhamman]

Good call. This function was coped from v2 but is now refactored along the lines of what you were suggesting.

[d-v-b]

I added a few minor comments, none of which are blockers. A few high-level thoughts:

A lot of the routines zarr.convenience seem strange to me, e.g. save has a very weird function signature that we should probably give a second thought when the dust settles from the v3 release. But changes like that are out of scope for this work.

I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?

This works for me.

I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

Definitely prefer what we have here over async_ones. I don't see any drawbacks to this organization, since presumably we want everything to rest on the async foundation, but I don't have much experience with libraries that expose async and sync apis.

[will-moore]

Hi, thanks for all the work on this.
I'm looking at using this from https://github.com/ome/ome-zarr-py and seeing this error when accessing via Dask:

 File "/Users/wmoore/Desktop/ZARR/ome-zarr-py/ome_zarr/io.py", line 177, in load
return da.from_zarr(self.__store, subpath)
File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/dask/array/core.py", line 3597, in from_zarr
z = zarr.Array(url, read_only=True, path=component, **kwargs)
TypeError: Array.__init__() got an unexpected keyword argument 'read_only'

Is read_only another argument to add to the list above at #1884 (comment) ?

[jhamman]

Hi @will-moore -- interesting to find out that Dask is using the array constructor directly. I would have expected to see zarr.array(...). This PR is covering the API methods formerly found in zarr.convenience and zarr.creation. We'll have to do a bit of work to handle the case you've shown here.

btw, zarr.array(url, read_only_true, path=component) should work with this PR.

[jhamman]

I'm planning to merge this in the morning unless I get further reviews.

[jhamman]

Marking as Work In Progress for now. Currently waiting on #1670 to merge.

Closes#1598
Fixes#1019

Open questions I could use feedback on:

1. I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?
2. I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

More soon.

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

cc @d-v-b, @aldenks

[jhamman]

This is ready for an initial review. To set expectations, I don't plan to address the following optional arguments in this PR:

• chunk_store
• cache_attrs
• synchronizer
• meta_array
• storage_options
• dimension_separator
• write_empty_chunks

These will need to be handled in later PRs.

[normanrz]

Suggested change

	MEMORY_ORDER=Literal["C", "F"]
	MemoryOrder=Literal["C", "F"]

[normanrz]

Can we use ChunkCoords here?

[jhamman]

yes, but maybe not for long. I went ahead and changed it but ShapeLike was intended to be int | tuple[int, ...]... Perhaps that will work for ChunkCoords too?

[d-v-b]

instead of mutating kwargs, can we copy it, and make changes to the copy, and return it?

[jhamman]

Good call. This function was coped from v2 but is now refactored along the lines of what you were suggesting.

[d-v-b]

I added a few minor comments, none of which are blockers. A few high-level thoughts:

A lot of the routines zarr.convenience seem strange to me, e.g. save has a very weird function signature that we should probably give a second thought when the dust settles from the v3 release. But changes like that are out of scope for this work.

I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?

This works for me.

I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

Definitely prefer what we have here over async_ones. I don't see any drawbacks to this organization, since presumably we want everything to rest on the async foundation, but I don't have much experience with libraries that expose async and sync apis.

[will-moore]

Hi, thanks for all the work on this.
I'm looking at using this from https://github.com/ome/ome-zarr-py and seeing this error when accessing via Dask:

 File "/Users/wmoore/Desktop/ZARR/ome-zarr-py/ome_zarr/io.py", line 177, in load
return da.from_zarr(self.__store, subpath)
File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/dask/array/core.py", line 3597, in from_zarr
z = zarr.Array(url, read_only=True, path=component, **kwargs)
TypeError: Array.__init__() got an unexpected keyword argument 'read_only'

Is read_only another argument to add to the list above at #1884 (comment) ?

[jhamman]

Hi @will-moore -- interesting to find out that Dask is using the array constructor directly. I would have expected to see zarr.array(...). This PR is covering the API methods formerly found in zarr.convenience and zarr.creation. We'll have to do a bit of work to handle the case you've shown here.

btw, zarr.array(url, read_only_true, path=component) should work with this PR.

[jhamman]

I'm planning to merge this in the morning unless I get further reviews.

[jhamman]

Marking as Work In Progress for now. Currently waiting on #1670 to merge.

Closes#1598
Fixes#1019

Open questions I could use feedback on:

1. I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?
2. I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

More soon.

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

cc @d-v-b, @aldenks

[jhamman]

This is ready for an initial review. To set expectations, I don't plan to address the following optional arguments in this PR:

• chunk_store
• cache_attrs
• synchronizer
• meta_array
• storage_options
• dimension_separator
• write_empty_chunks

These will need to be handled in later PRs.

[normanrz]

Suggested change

	MEMORY_ORDER=Literal["C", "F"]
	MemoryOrder=Literal["C", "F"]

[normanrz]

Can we use ChunkCoords here?

[jhamman]

yes, but maybe not for long. I went ahead and changed it but ShapeLike was intended to be int | tuple[int, ...]... Perhaps that will work for ChunkCoords too?

[d-v-b]

instead of mutating kwargs, can we copy it, and make changes to the copy, and return it?

[jhamman]

Good call. This function was coped from v2 but is now refactored along the lines of what you were suggesting.

[d-v-b]

I added a few minor comments, none of which are blockers. A few high-level thoughts:

A lot of the routines zarr.convenience seem strange to me, e.g. save has a very weird function signature that we should probably give a second thought when the dust settles from the v3 release. But changes like that are out of scope for this work.

I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?

This works for me.

I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

Definitely prefer what we have here over async_ones. I don't see any drawbacks to this organization, since presumably we want everything to rest on the async foundation, but I don't have much experience with libraries that expose async and sync apis.

[will-moore]

Hi, thanks for all the work on this.
I'm looking at using this from https://github.com/ome/ome-zarr-py and seeing this error when accessing via Dask:

 File "/Users/wmoore/Desktop/ZARR/ome-zarr-py/ome_zarr/io.py", line 177, in load
return da.from_zarr(self.__store, subpath)
File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/dask/array/core.py", line 3597, in from_zarr
z = zarr.Array(url, read_only=True, path=component, **kwargs)
TypeError: Array.__init__() got an unexpected keyword argument 'read_only'

Is read_only another argument to add to the list above at #1884 (comment) ?

[jhamman]

Hi @will-moore -- interesting to find out that Dask is using the array constructor directly. I would have expected to see zarr.array(...). This PR is covering the API methods formerly found in zarr.convenience and zarr.creation. We'll have to do a bit of work to handle the case you've shown here.

btw, zarr.array(url, read_only_true, path=component) should work with this PR.

[jhamman]

I'm planning to merge this in the morning unless I get further reviews.

[jhamman]

Marking as Work In Progress for now. Currently waiting on #1670 to merge.

Closes#1598
Fixes#1019

Open questions I could use feedback on:

1. I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?
2. I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

More soon.

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

cc @d-v-b, @aldenks

[jhamman]

This is ready for an initial review. To set expectations, I don't plan to address the following optional arguments in this PR:

• chunk_store
• cache_attrs
• synchronizer
• meta_array
• storage_options
• dimension_separator
• write_empty_chunks

These will need to be handled in later PRs.

[normanrz]

Suggested change

	MEMORY_ORDER=Literal["C", "F"]
	MemoryOrder=Literal["C", "F"]

[normanrz]

Can we use ChunkCoords here?

[jhamman]

yes, but maybe not for long. I went ahead and changed it but ShapeLike was intended to be int | tuple[int, ...]... Perhaps that will work for ChunkCoords too?

[d-v-b]

instead of mutating kwargs, can we copy it, and make changes to the copy, and return it?

[jhamman]

Good call. This function was coped from v2 but is now refactored along the lines of what you were suggesting.

[d-v-b]

I added a few minor comments, none of which are blockers. A few high-level thoughts:

A lot of the routines zarr.convenience seem strange to me, e.g. save has a very weird function signature that we should probably give a second thought when the dust settles from the v3 release. But changes like that are out of scope for this work.

I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?

This works for me.

I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

Definitely prefer what we have here over async_ones. I don't see any drawbacks to this organization, since presumably we want everything to rest on the async foundation, but I don't have much experience with libraries that expose async and sync apis.

[will-moore]

Hi, thanks for all the work on this.
I'm looking at using this from https://github.com/ome/ome-zarr-py and seeing this error when accessing via Dask:

 File "/Users/wmoore/Desktop/ZARR/ome-zarr-py/ome_zarr/io.py", line 177, in load
return da.from_zarr(self.__store, subpath)
File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/dask/array/core.py", line 3597, in from_zarr
z = zarr.Array(url, read_only=True, path=component, **kwargs)
TypeError: Array.__init__() got an unexpected keyword argument 'read_only'

Is read_only another argument to add to the list above at #1884 (comment) ?

[jhamman]

Hi @will-moore -- interesting to find out that Dask is using the array constructor directly. I would have expected to see zarr.array(...). This PR is covering the API methods formerly found in zarr.convenience and zarr.creation. We'll have to do a bit of work to handle the case you've shown here.

btw, zarr.array(url, read_only_true, path=component) should work with this PR.

[jhamman]

I'm planning to merge this in the morning unless I get further reviews.

[jhamman]

Marking as Work In Progress for now. Currently waiting on #1670 to merge.

Closes#1598
Fixes#1019

Open questions I could use feedback on:

1. I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?
2. I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

More soon.

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

cc @d-v-b, @aldenks

[jhamman]

This is ready for an initial review. To set expectations, I don't plan to address the following optional arguments in this PR:

• chunk_store
• cache_attrs
• synchronizer
• meta_array
• storage_options
• dimension_separator
• write_empty_chunks

These will need to be handled in later PRs.

[normanrz]

Suggested change

	MEMORY_ORDER=Literal["C", "F"]
	MemoryOrder=Literal["C", "F"]

[normanrz]

Can we use ChunkCoords here?

[jhamman]

yes, but maybe not for long. I went ahead and changed it but ShapeLike was intended to be int | tuple[int, ...]... Perhaps that will work for ChunkCoords too?

[d-v-b]

instead of mutating kwargs, can we copy it, and make changes to the copy, and return it?

[jhamman]

Good call. This function was coped from v2 but is now refactored along the lines of what you were suggesting.

[d-v-b]

I added a few minor comments, none of which are blockers. A few high-level thoughts:

A lot of the routines zarr.convenience seem strange to me, e.g. save has a very weird function signature that we should probably give a second thought when the dust settles from the v3 release. But changes like that are out of scope for this work.

I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?

This works for me.

I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

Definitely prefer what we have here over async_ones. I don't see any drawbacks to this organization, since presumably we want everything to rest on the async foundation, but I don't have much experience with libraries that expose async and sync apis.

[will-moore]

Hi, thanks for all the work on this.
I'm looking at using this from https://github.com/ome/ome-zarr-py and seeing this error when accessing via Dask:

 File "/Users/wmoore/Desktop/ZARR/ome-zarr-py/ome_zarr/io.py", line 177, in load
return da.from_zarr(self.__store, subpath)
File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/dask/array/core.py", line 3597, in from_zarr
z = zarr.Array(url, read_only=True, path=component, **kwargs)
TypeError: Array.__init__() got an unexpected keyword argument 'read_only'

Is read_only another argument to add to the list above at #1884 (comment) ?

[jhamman]

Hi @will-moore -- interesting to find out that Dask is using the array constructor directly. I would have expected to see zarr.array(...). This PR is covering the API methods formerly found in zarr.convenience and zarr.creation. We'll have to do a bit of work to handle the case you've shown here.

btw, zarr.array(url, read_only_true, path=component) should work with this PR.

[jhamman]

I'm planning to merge this in the morning unless I get further reviews.

[jhamman]

Marking as Work In Progress for now. Currently waiting on #1670 to merge.

Closes#1598
Fixes#1019

Open questions I could use feedback on:

1. I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?
2. I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

More soon.

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

cc @d-v-b, @aldenks

[jhamman]

This is ready for an initial review. To set expectations, I don't plan to address the following optional arguments in this PR:

• chunk_store
• cache_attrs
• synchronizer
• meta_array
• storage_options
• dimension_separator
• write_empty_chunks

These will need to be handled in later PRs.

[normanrz]

Suggested change

	MEMORY_ORDER=Literal["C", "F"]
	MemoryOrder=Literal["C", "F"]

[normanrz]

Can we use ChunkCoords here?

[jhamman]

yes, but maybe not for long. I went ahead and changed it but ShapeLike was intended to be int | tuple[int, ...]... Perhaps that will work for ChunkCoords too?

[d-v-b]

instead of mutating kwargs, can we copy it, and make changes to the copy, and return it?

[jhamman]

Good call. This function was coped from v2 but is now refactored along the lines of what you were suggesting.

[d-v-b]

I added a few minor comments, none of which are blockers. A few high-level thoughts:

A lot of the routines zarr.convenience seem strange to me, e.g. save has a very weird function signature that we should probably give a second thought when the dust settles from the v3 release. But changes like that are out of scope for this work.

I've combined zarr.creation and zarr.convenience into a common API module. This makes sense to me because they are all basically wrappers that help us construct Group/Array objects. What people think about this?

This works for me.

I've created parallel synchronous and asynchronous modules with basically identical APIs. All the synchronous functions simply call their asynchronous cousin. I like this much more than having two versions of each function (e.g. ones and async_ones). Thoughts?

Definitely prefer what we have here over async_ones. I don't see any drawbacks to this organization, since presumably we want everything to rest on the async foundation, but I don't have much experience with libraries that expose async and sync apis.

[will-moore]

Hi, thanks for all the work on this.
I'm looking at using this from https://github.com/ome/ome-zarr-py and seeing this error when accessing via Dask:

 File "/Users/wmoore/Desktop/ZARR/ome-zarr-py/ome_zarr/io.py", line 177, in load
return da.from_zarr(self.__store, subpath)
File "/Users/wmoore/opt/anaconda3/envs/zarr_v3/lib/python3.10/site-packages/dask/array/core.py", line 3597, in from_zarr
z = zarr.Array(url, read_only=True, path=component, **kwargs)
TypeError: Array.__init__() got an unexpected keyword argument 'read_only'

Is read_only another argument to add to the list above at #1884 (comment) ?

[jhamman]

Hi @will-moore -- interesting to find out that Dask is using the array constructor directly. I would have expected to see zarr.array(...). This PR is covering the API methods formerly found in zarr.convenience and zarr.creation. We'll have to do a bit of work to handle the case you've shown here.

btw, zarr.array(url, read_only_true, path=component) should work with this PR.

[jhamman]

I'm planning to merge this in the morning unless I get further reviews.