Extensible codecs for V3 by normanrz · Pull Request #1588 · zarr-developers/zarr-python

normanrz · 2023-12-05T13:31:40Z

Makes codecs extensible in the v3 branch by adding a codec registry.

No code changes to Zarrita were made.

pep8speaks · 2023-12-05T17:16:32Z

Hello @normanrz! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file zarr/tests/test_codecs_v3.py:

Line 76:20: E203 whitespace before ':'
Line 76:46: E203 whitespace before ':'
Line 76:72: E203 whitespace before ':'
Line 214:20: E203 whitespace before ':'
Line 214:46: E203 whitespace before ':'
Line 214:72: E203 whitespace before ':'

In the file zarr/v3/store.py:

Line 301:44: E203 whitespace before ':'
Line 313:30: E203 whitespace before ':'

Comment last updated at 2023-12-06 10:51:06 UTC

jhamman

I mostly reviewed the entrypoints stuff since the rest seemed to be reorganization / porting from zarrita. Looks really nice!

jhamman · 2023-12-06T10:23:38Z

@@ -59,24 +53,8 @@ def _structure_chunk_key_encoding_metadata(d: Dict[str, Any], _t) -> ChunkKeyEnc
 )

 def _structure_codec_metadata(d: Dict[str, Any], _t=None) -> CodecMetadata:


not for now but it would be nice if the codec metadata was a TypedDict so we could expect for keys like "name".

name is really the only shared attribute among all codecs. Even configuration is optional. I don't think there'd be much value in a TypedDict. This piece of code is in the middle of the deserialization; lots of untyped magic happening here anyways :-)

Co-authored-by: Joe Hamman <jhamman1@gmail.com>

normanrz · 2023-12-06T17:37:16Z

+@frozen
+class _AsyncArrayProxy:
+ array: AsyncArray
+
+ def __getitem__(self, selection: Selection) -> _AsyncArraySelectionProxy:
+ return _AsyncArraySelectionProxy(self.array, selection)
+
+
+@frozen
+class _AsyncArraySelectionProxy:
+ array: AsyncArray
+ selection: Selection
+
+ async def get(self) -> np.ndarray:
+ return await self.array.getitem(self.selection)
+
+ async def set(self, value: np.ndarray):
+ return await self.array.setitem(self.selection, value)


@jhamman @d-v-b What do you think about this making it into the AsyncArray? It allows for writing something like this
await _AsyncArrayProxy(array)[:10, 10:].get()
instead of
await array.getitem((slice(None, 10), slice(10, None)))

* Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06 No code changes to Zarrita were made. * apply zarr lint rules * zarrita -> v3 * v3/abc [wip] * use abcs plus implementation notes * working on making codecs extensible * adds index_location * adds support for codec entry points * adds tests from zarrita * fixes types * Apply suggestions from code review Co-authored-by: Joe Hamman <jhamman1@gmail.com> * remove test codec from pyproject.toml --------- Co-authored-by: Joseph Hamman <joe@earthmover.io> Co-authored-by: Joe Hamman <jhamman1@gmail.com>

normanrz · 2023-12-05T13:31:40Z

Makes codecs extensible in the v3 branch by adding a codec registry.

No code changes to Zarrita were made.

pep8speaks · 2023-12-05T17:16:32Z

Hello @normanrz! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file zarr/tests/test_codecs_v3.py:

Line 76:20: E203 whitespace before ':'
Line 76:46: E203 whitespace before ':'
Line 76:72: E203 whitespace before ':'
Line 214:20: E203 whitespace before ':'
Line 214:46: E203 whitespace before ':'
Line 214:72: E203 whitespace before ':'

In the file zarr/v3/store.py:

Line 301:44: E203 whitespace before ':'
Line 313:30: E203 whitespace before ':'

Comment last updated at 2023-12-06 10:51:06 UTC

jhamman

I mostly reviewed the entrypoints stuff since the rest seemed to be reorganization / porting from zarrita. Looks really nice!

jhamman · 2023-12-06T10:23:38Z

@@ -59,24 +53,8 @@ def _structure_chunk_key_encoding_metadata(d: Dict[str, Any], _t) -> ChunkKeyEnc
 )

 def _structure_codec_metadata(d: Dict[str, Any], _t=None) -> CodecMetadata:


not for now but it would be nice if the codec metadata was a TypedDict so we could expect for keys like "name".

name is really the only shared attribute among all codecs. Even configuration is optional. I don't think there'd be much value in a TypedDict. This piece of code is in the middle of the deserialization; lots of untyped magic happening here anyways :-)

Co-authored-by: Joe Hamman <jhamman1@gmail.com>

normanrz · 2023-12-06T17:37:16Z

+@frozen
+class _AsyncArrayProxy:
+ array: AsyncArray
+
+ def __getitem__(self, selection: Selection) -> _AsyncArraySelectionProxy:
+ return _AsyncArraySelectionProxy(self.array, selection)
+
+
+@frozen
+class _AsyncArraySelectionProxy:
+ array: AsyncArray
+ selection: Selection
+
+ async def get(self) -> np.ndarray:
+ return await self.array.getitem(self.selection)
+
+ async def set(self, value: np.ndarray):
+ return await self.array.setitem(self.selection, value)


@jhamman @d-v-b What do you think about this making it into the AsyncArray? It allows for writing something like this
await _AsyncArrayProxy(array)[:10, 10:].get()
instead of
await array.getitem((slice(None, 10), slice(10, None)))

* Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06 No code changes to Zarrita were made. * apply zarr lint rules * zarrita -> v3 * v3/abc [wip] * use abcs plus implementation notes * working on making codecs extensible * adds index_location * adds support for codec entry points * adds tests from zarrita * fixes types * Apply suggestions from code review Co-authored-by: Joe Hamman <jhamman1@gmail.com> * remove test codec from pyproject.toml --------- Co-authored-by: Joseph Hamman <joe@earthmover.io> Co-authored-by: Joe Hamman <jhamman1@gmail.com>

normanrz · 2023-12-05T13:31:40Z

Makes codecs extensible in the v3 branch by adding a codec registry.

No code changes to Zarrita were made.

pep8speaks · 2023-12-05T17:16:32Z

Hello @normanrz! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file zarr/tests/test_codecs_v3.py:

Line 76:20: E203 whitespace before ':'
Line 76:46: E203 whitespace before ':'
Line 76:72: E203 whitespace before ':'
Line 214:20: E203 whitespace before ':'
Line 214:46: E203 whitespace before ':'
Line 214:72: E203 whitespace before ':'

In the file zarr/v3/store.py:

Line 301:44: E203 whitespace before ':'
Line 313:30: E203 whitespace before ':'

Comment last updated at 2023-12-06 10:51:06 UTC

jhamman

I mostly reviewed the entrypoints stuff since the rest seemed to be reorganization / porting from zarrita. Looks really nice!

jhamman · 2023-12-06T10:23:38Z

@@ -59,24 +53,8 @@ def _structure_chunk_key_encoding_metadata(d: Dict[str, Any], _t) -> ChunkKeyEnc
 )

 def _structure_codec_metadata(d: Dict[str, Any], _t=None) -> CodecMetadata:


not for now but it would be nice if the codec metadata was a TypedDict so we could expect for keys like "name".

name is really the only shared attribute among all codecs. Even configuration is optional. I don't think there'd be much value in a TypedDict. This piece of code is in the middle of the deserialization; lots of untyped magic happening here anyways :-)

Co-authored-by: Joe Hamman <jhamman1@gmail.com>

normanrz · 2023-12-06T17:37:16Z

+@frozen
+class _AsyncArrayProxy:
+ array: AsyncArray
+
+ def __getitem__(self, selection: Selection) -> _AsyncArraySelectionProxy:
+ return _AsyncArraySelectionProxy(self.array, selection)
+
+
+@frozen
+class _AsyncArraySelectionProxy:
+ array: AsyncArray
+ selection: Selection
+
+ async def get(self) -> np.ndarray:
+ return await self.array.getitem(self.selection)
+
+ async def set(self, value: np.ndarray):
+ return await self.array.setitem(self.selection, value)


@jhamman @d-v-b What do you think about this making it into the AsyncArray? It allows for writing something like this
await _AsyncArrayProxy(array)[:10, 10:].get()
instead of
await array.getitem((slice(None, 10), slice(10, None)))

* Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06 No code changes to Zarrita were made. * apply zarr lint rules * zarrita -> v3 * v3/abc [wip] * use abcs plus implementation notes * working on making codecs extensible * adds index_location * adds support for codec entry points * adds tests from zarrita * fixes types * Apply suggestions from code review Co-authored-by: Joe Hamman <jhamman1@gmail.com> * remove test codec from pyproject.toml --------- Co-authored-by: Joseph Hamman <joe@earthmover.io> Co-authored-by: Joe Hamman <jhamman1@gmail.com>

normanrz · 2023-12-05T13:31:40Z

Makes codecs extensible in the v3 branch by adding a codec registry.

No code changes to Zarrita were made.

pep8speaks · 2023-12-05T17:16:32Z

Hello @normanrz! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file zarr/tests/test_codecs_v3.py:

Line 76:20: E203 whitespace before ':'
Line 76:46: E203 whitespace before ':'
Line 76:72: E203 whitespace before ':'
Line 214:20: E203 whitespace before ':'
Line 214:46: E203 whitespace before ':'
Line 214:72: E203 whitespace before ':'

In the file zarr/v3/store.py:

Line 301:44: E203 whitespace before ':'
Line 313:30: E203 whitespace before ':'

Comment last updated at 2023-12-06 10:51:06 UTC

jhamman

I mostly reviewed the entrypoints stuff since the rest seemed to be reorganization / porting from zarrita. Looks really nice!

jhamman · 2023-12-06T10:23:38Z

@@ -59,24 +53,8 @@ def _structure_chunk_key_encoding_metadata(d: Dict[str, Any], _t) -> ChunkKeyEnc
 )

 def _structure_codec_metadata(d: Dict[str, Any], _t=None) -> CodecMetadata:


not for now but it would be nice if the codec metadata was a TypedDict so we could expect for keys like "name".

name is really the only shared attribute among all codecs. Even configuration is optional. I don't think there'd be much value in a TypedDict. This piece of code is in the middle of the deserialization; lots of untyped magic happening here anyways :-)

Co-authored-by: Joe Hamman <jhamman1@gmail.com>

normanrz · 2023-12-06T17:37:16Z

+@frozen
+class _AsyncArrayProxy:
+ array: AsyncArray
+
+ def __getitem__(self, selection: Selection) -> _AsyncArraySelectionProxy:
+ return _AsyncArraySelectionProxy(self.array, selection)
+
+
+@frozen
+class _AsyncArraySelectionProxy:
+ array: AsyncArray
+ selection: Selection
+
+ async def get(self) -> np.ndarray:
+ return await self.array.getitem(self.selection)
+
+ async def set(self, value: np.ndarray):
+ return await self.array.setitem(self.selection, value)


@jhamman @d-v-b What do you think about this making it into the AsyncArray? It allows for writing something like this
await _AsyncArrayProxy(array)[:10, 10:].get()
instead of
await array.getitem((slice(None, 10), slice(10, None)))

* Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06 No code changes to Zarrita were made. * apply zarr lint rules * zarrita -> v3 * v3/abc [wip] * use abcs plus implementation notes * working on making codecs extensible * adds index_location * adds support for codec entry points * adds tests from zarrita * fixes types * Apply suggestions from code review Co-authored-by: Joe Hamman <jhamman1@gmail.com> * remove test codec from pyproject.toml --------- Co-authored-by: Joseph Hamman <joe@earthmover.io> Co-authored-by: Joe Hamman <jhamman1@gmail.com>

normanrz · 2023-12-05T13:31:40Z

Makes codecs extensible in the v3 branch by adding a codec registry.

No code changes to Zarrita were made.

pep8speaks · 2023-12-05T17:16:32Z

Hello @normanrz! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file zarr/tests/test_codecs_v3.py:

Line 76:20: E203 whitespace before ':'
Line 76:46: E203 whitespace before ':'
Line 76:72: E203 whitespace before ':'
Line 214:20: E203 whitespace before ':'
Line 214:46: E203 whitespace before ':'
Line 214:72: E203 whitespace before ':'

In the file zarr/v3/store.py:

Line 301:44: E203 whitespace before ':'
Line 313:30: E203 whitespace before ':'

Comment last updated at 2023-12-06 10:51:06 UTC

jhamman

I mostly reviewed the entrypoints stuff since the rest seemed to be reorganization / porting from zarrita. Looks really nice!

jhamman · 2023-12-06T10:23:38Z

@@ -59,24 +53,8 @@ def _structure_chunk_key_encoding_metadata(d: Dict[str, Any], _t) -> ChunkKeyEnc
 )

 def _structure_codec_metadata(d: Dict[str, Any], _t=None) -> CodecMetadata:


not for now but it would be nice if the codec metadata was a TypedDict so we could expect for keys like "name".

name is really the only shared attribute among all codecs. Even configuration is optional. I don't think there'd be much value in a TypedDict. This piece of code is in the middle of the deserialization; lots of untyped magic happening here anyways :-)

Co-authored-by: Joe Hamman <jhamman1@gmail.com>

normanrz · 2023-12-06T17:37:16Z

+@frozen
+class _AsyncArrayProxy:
+ array: AsyncArray
+
+ def __getitem__(self, selection: Selection) -> _AsyncArraySelectionProxy:
+ return _AsyncArraySelectionProxy(self.array, selection)
+
+
+@frozen
+class _AsyncArraySelectionProxy:
+ array: AsyncArray
+ selection: Selection
+
+ async def get(self) -> np.ndarray:
+ return await self.array.getitem(self.selection)
+
+ async def set(self, value: np.ndarray):
+ return await self.array.setitem(self.selection, value)


@jhamman @d-v-b What do you think about this making it into the AsyncArray? It allows for writing something like this
await _AsyncArrayProxy(array)[:10, 10:].get()
instead of
await array.getitem((slice(None, 10), slice(10, None)))

* Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06 No code changes to Zarrita were made. * apply zarr lint rules * zarrita -> v3 * v3/abc [wip] * use abcs plus implementation notes * working on making codecs extensible * adds index_location * adds support for codec entry points * adds tests from zarrita * fixes types * Apply suggestions from code review Co-authored-by: Joe Hamman <jhamman1@gmail.com> * remove test codec from pyproject.toml --------- Co-authored-by: Joseph Hamman <joe@earthmover.io> Co-authored-by: Joe Hamman <jhamman1@gmail.com>

normanrz · 2023-12-05T13:31:40Z

Makes codecs extensible in the v3 branch by adding a codec registry.

No code changes to Zarrita were made.

pep8speaks · 2023-12-05T17:16:32Z

Hello @normanrz! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file zarr/tests/test_codecs_v3.py:

Line 76:20: E203 whitespace before ':'
Line 76:46: E203 whitespace before ':'
Line 76:72: E203 whitespace before ':'
Line 214:20: E203 whitespace before ':'
Line 214:46: E203 whitespace before ':'
Line 214:72: E203 whitespace before ':'

In the file zarr/v3/store.py:

Line 301:44: E203 whitespace before ':'
Line 313:30: E203 whitespace before ':'

Comment last updated at 2023-12-06 10:51:06 UTC

jhamman

I mostly reviewed the entrypoints stuff since the rest seemed to be reorganization / porting from zarrita. Looks really nice!

jhamman · 2023-12-06T10:23:38Z

@@ -59,24 +53,8 @@ def _structure_chunk_key_encoding_metadata(d: Dict[str, Any], _t) -> ChunkKeyEnc
 )

 def _structure_codec_metadata(d: Dict[str, Any], _t=None) -> CodecMetadata:


not for now but it would be nice if the codec metadata was a TypedDict so we could expect for keys like "name".

name is really the only shared attribute among all codecs. Even configuration is optional. I don't think there'd be much value in a TypedDict. This piece of code is in the middle of the deserialization; lots of untyped magic happening here anyways :-)

Co-authored-by: Joe Hamman <jhamman1@gmail.com>

normanrz · 2023-12-06T17:37:16Z

+@frozen
+class _AsyncArrayProxy:
+ array: AsyncArray
+
+ def __getitem__(self, selection: Selection) -> _AsyncArraySelectionProxy:
+ return _AsyncArraySelectionProxy(self.array, selection)
+
+
+@frozen
+class _AsyncArraySelectionProxy:
+ array: AsyncArray
+ selection: Selection
+
+ async def get(self) -> np.ndarray:
+ return await self.array.getitem(self.selection)
+
+ async def set(self, value: np.ndarray):
+ return await self.array.setitem(self.selection, value)


@jhamman @d-v-b What do you think about this making it into the AsyncArray? It allows for writing something like this
await _AsyncArrayProxy(array)[:10, 10:].get()
instead of
await array.getitem((slice(None, 10), slice(10, None)))

* Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06 No code changes to Zarrita were made. * apply zarr lint rules * zarrita -> v3 * v3/abc [wip] * use abcs plus implementation notes * working on making codecs extensible * adds index_location * adds support for codec entry points * adds tests from zarrita * fixes types * Apply suggestions from code review Co-authored-by: Joe Hamman <jhamman1@gmail.com> * remove test codec from pyproject.toml --------- Co-authored-by: Joseph Hamman <joe@earthmover.io> Co-authored-by: Joe Hamman <jhamman1@gmail.com>

normanrz · 2023-12-05T13:31:40Z

Makes codecs extensible in the v3 branch by adding a codec registry.

No code changes to Zarrita were made.

pep8speaks · 2023-12-05T17:16:32Z

Hello @normanrz! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file zarr/tests/test_codecs_v3.py:

Line 76:20: E203 whitespace before ':'
Line 76:46: E203 whitespace before ':'
Line 76:72: E203 whitespace before ':'
Line 214:20: E203 whitespace before ':'
Line 214:46: E203 whitespace before ':'
Line 214:72: E203 whitespace before ':'

In the file zarr/v3/store.py:

Line 301:44: E203 whitespace before ':'
Line 313:30: E203 whitespace before ':'

Comment last updated at 2023-12-06 10:51:06 UTC

jhamman

I mostly reviewed the entrypoints stuff since the rest seemed to be reorganization / porting from zarrita. Looks really nice!

jhamman · 2023-12-06T10:23:38Z

@@ -59,24 +53,8 @@ def _structure_chunk_key_encoding_metadata(d: Dict[str, Any], _t) -> ChunkKeyEnc
 )

 def _structure_codec_metadata(d: Dict[str, Any], _t=None) -> CodecMetadata:


not for now but it would be nice if the codec metadata was a TypedDict so we could expect for keys like "name".

name is really the only shared attribute among all codecs. Even configuration is optional. I don't think there'd be much value in a TypedDict. This piece of code is in the middle of the deserialization; lots of untyped magic happening here anyways :-)

Co-authored-by: Joe Hamman <jhamman1@gmail.com>

normanrz · 2023-12-06T17:37:16Z

+@frozen
+class _AsyncArrayProxy:
+ array: AsyncArray
+
+ def __getitem__(self, selection: Selection) -> _AsyncArraySelectionProxy:
+ return _AsyncArraySelectionProxy(self.array, selection)
+
+
+@frozen
+class _AsyncArraySelectionProxy:
+ array: AsyncArray
+ selection: Selection
+
+ async def get(self) -> np.ndarray:
+ return await self.array.getitem(self.selection)
+
+ async def set(self, value: np.ndarray):
+ return await self.array.setitem(self.selection, value)


@jhamman @d-v-b What do you think about this making it into the AsyncArray? It allows for writing something like this
await _AsyncArrayProxy(array)[:10, 10:].get()
instead of
await array.getitem((slice(None, 10), slice(10, None)))

* Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06 No code changes to Zarrita were made. * apply zarr lint rules * zarrita -> v3 * v3/abc [wip] * use abcs plus implementation notes * working on making codecs extensible * adds index_location * adds support for codec entry points * adds tests from zarrita * fixes types * Apply suggestions from code review Co-authored-by: Joe Hamman <jhamman1@gmail.com> * remove test codec from pyproject.toml --------- Co-authored-by: Joseph Hamman <joe@earthmover.io> Co-authored-by: Joe Hamman <jhamman1@gmail.com>

normanrz · 2023-12-05T13:31:40Z

Makes codecs extensible in the v3 branch by adding a codec registry.

No code changes to Zarrita were made.

pep8speaks · 2023-12-05T17:16:32Z

Hello @normanrz! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file zarr/tests/test_codecs_v3.py:

Line 76:20: E203 whitespace before ':'
Line 76:46: E203 whitespace before ':'
Line 76:72: E203 whitespace before ':'
Line 214:20: E203 whitespace before ':'
Line 214:46: E203 whitespace before ':'
Line 214:72: E203 whitespace before ':'

In the file zarr/v3/store.py:

Line 301:44: E203 whitespace before ':'
Line 313:30: E203 whitespace before ':'

Comment last updated at 2023-12-06 10:51:06 UTC

jhamman

I mostly reviewed the entrypoints stuff since the rest seemed to be reorganization / porting from zarrita. Looks really nice!

jhamman · 2023-12-06T10:23:38Z

@@ -59,24 +53,8 @@ def _structure_chunk_key_encoding_metadata(d: Dict[str, Any], _t) -> ChunkKeyEnc
 )

 def _structure_codec_metadata(d: Dict[str, Any], _t=None) -> CodecMetadata:


not for now but it would be nice if the codec metadata was a TypedDict so we could expect for keys like "name".

name is really the only shared attribute among all codecs. Even configuration is optional. I don't think there'd be much value in a TypedDict. This piece of code is in the middle of the deserialization; lots of untyped magic happening here anyways :-)

Co-authored-by: Joe Hamman <jhamman1@gmail.com>

normanrz · 2023-12-06T17:37:16Z

+@frozen
+class _AsyncArrayProxy:
+ array: AsyncArray
+
+ def __getitem__(self, selection: Selection) -> _AsyncArraySelectionProxy:
+ return _AsyncArraySelectionProxy(self.array, selection)
+
+
+@frozen
+class _AsyncArraySelectionProxy:
+ array: AsyncArray
+ selection: Selection
+
+ async def get(self) -> np.ndarray:
+ return await self.array.getitem(self.selection)
+
+ async def set(self, value: np.ndarray):
+ return await self.array.setitem(self.selection, value)


@jhamman @d-v-b What do you think about this making it into the AsyncArray? It allows for writing something like this
await _AsyncArrayProxy(array)[:10, 10:].get()
instead of
await array.getitem((slice(None, 10), slice(10, None)))

* Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06 No code changes to Zarrita were made. * apply zarr lint rules * zarrita -> v3 * v3/abc [wip] * use abcs plus implementation notes * working on making codecs extensible * adds index_location * adds support for codec entry points * adds tests from zarrita * fixes types * Apply suggestions from code review Co-authored-by: Joe Hamman <jhamman1@gmail.com> * remove test codec from pyproject.toml --------- Co-authored-by: Joseph Hamman <joe@earthmover.io> Co-authored-by: Joe Hamman <jhamman1@gmail.com>

jhammanand others added 10 commits November 3, 2023 12:36

Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06
55b47b1
No code changes to Zarrita were made.

apply zarr lint rules
a31a4e5

zarrita -> v3
15f667c

v3/abc [wip]
c3f8764

use abcs plus implementation notes
0864187

working on making codecs extensible
78d0bc0

merge
8b86afa

adds index_location
4fac528

adds support for codec entry points
fcf79b6

adds tests from zarrita
9f97282

fixes types
f8bab5b

jhamman added the V3-compat label Dec 5, 2023

jhamman approved these changes Dec 6, 2023
View reviewed changes

normanrzand others added 2 commits December 6, 2023 11:34

Apply suggestions from code review
6c3e40a
Co-authored-by: Joe Hamman <jhamman1@gmail.com>

remove test codec from pyproject.toml
9749f25

jhamman merged commit 1a98886 into zarr-developers:v3Dec 6, 2023

normanrz commented Dec 6, 2023
View reviewed changes

This was referenced Dec 7, 2023
Implement Variable Chunking (ZEP003) in V3 #1595
Closed
doc(v3): add v3 roadmap and design document #1583
Merged

jhamman mentioned this pull request Jan 24, 2024
Supporting Zarr-Python 3 Codec API zarr-developers/numcodecs#502
Closed

jhamman added this to the 3.0.0.alpha milestone Apr 22, 2024

sharkinsspatial mentioned this pull request Jul 24, 2024
Non-kerchunk backend for HDF5/netcdf4 files. zarr-developers/VirtualiZarr#87
Merged

dstansby mentioned this pull request Jan 26, 2025
Difference in behavior between 2.x and 3.x using identical compressor settings #2766
Closed

jhammanand others added 10 commits November 3, 2023 12:36

Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06
55b47b1
No code changes to Zarrita were made.

apply zarr lint rules
a31a4e5

zarrita -> v3
15f667c

v3/abc [wip]
c3f8764

use abcs plus implementation notes
0864187

working on making codecs extensible
78d0bc0

merge
8b86afa

adds index_location
4fac528

adds support for codec entry points
fcf79b6

adds tests from zarrita
9f97282

fixes types
f8bab5b

jhamman added the V3-compat label Dec 5, 2023

jhamman approved these changes Dec 6, 2023
View reviewed changes

normanrzand others added 2 commits December 6, 2023 11:34

Apply suggestions from code review
6c3e40a
Co-authored-by: Joe Hamman <jhamman1@gmail.com>

remove test codec from pyproject.toml
9749f25

jhamman merged commit 1a98886 into zarr-developers:v3Dec 6, 2023

normanrz commented Dec 6, 2023
View reviewed changes

This was referenced Dec 7, 2023
Implement Variable Chunking (ZEP003) in V3 #1595
Closed
doc(v3): add v3 roadmap and design document #1583
Merged

jhamman mentioned this pull request Jan 24, 2024
Supporting Zarr-Python 3 Codec API zarr-developers/numcodecs#502
Closed

jhamman added this to the 3.0.0.alpha milestone Apr 22, 2024

sharkinsspatial mentioned this pull request Jul 24, 2024
Non-kerchunk backend for HDF5/netcdf4 files. zarr-developers/VirtualiZarr#87
Merged

dstansby mentioned this pull request Jan 26, 2025
Difference in behavior between 2.x and 3.x using identical compressor settings #2766
Closed

jhammanand others added 10 commits November 3, 2023 12:36

Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06
55b47b1
No code changes to Zarrita were made.

apply zarr lint rules
a31a4e5

zarrita -> v3
15f667c

v3/abc [wip]
c3f8764

use abcs plus implementation notes
0864187

working on making codecs extensible
78d0bc0

merge
8b86afa

adds index_location
4fac528

adds support for codec entry points
fcf79b6

adds tests from zarrita
9f97282

fixes types
f8bab5b

jhamman added the V3-compat label Dec 5, 2023

jhamman approved these changes Dec 6, 2023
View reviewed changes

normanrzand others added 2 commits December 6, 2023 11:34

Apply suggestions from code review
6c3e40a
Co-authored-by: Joe Hamman <jhamman1@gmail.com>

remove test codec from pyproject.toml
9749f25

jhamman merged commit 1a98886 into zarr-developers:v3Dec 6, 2023

normanrz commented Dec 6, 2023
View reviewed changes

This was referenced Dec 7, 2023
Implement Variable Chunking (ZEP003) in V3 #1595
Closed
doc(v3): add v3 roadmap and design document #1583
Merged

jhamman mentioned this pull request Jan 24, 2024
Supporting Zarr-Python 3 Codec API zarr-developers/numcodecs#502
Closed

jhamman added this to the 3.0.0.alpha milestone Apr 22, 2024

sharkinsspatial mentioned this pull request Jul 24, 2024
Non-kerchunk backend for HDF5/netcdf4 files. zarr-developers/VirtualiZarr#87
Merged

dstansby mentioned this pull request Jan 26, 2025
Difference in behavior between 2.x and 3.x using identical compressor settings #2766
Closed

jhammanand others added 10 commits November 3, 2023 12:36

Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06
55b47b1
No code changes to Zarrita were made.

apply zarr lint rules
a31a4e5

zarrita -> v3
15f667c

v3/abc [wip]
c3f8764

use abcs plus implementation notes
0864187

working on making codecs extensible
78d0bc0

merge
8b86afa

adds index_location
4fac528

adds support for codec entry points
fcf79b6

adds tests from zarrita
9f97282

fixes types
f8bab5b

jhamman added the V3-compat label Dec 5, 2023

jhamman approved these changes Dec 6, 2023
View reviewed changes

normanrzand others added 2 commits December 6, 2023 11:34

Apply suggestions from code review
6c3e40a
Co-authored-by: Joe Hamman <jhamman1@gmail.com>

remove test codec from pyproject.toml
9749f25

jhamman merged commit 1a98886 into zarr-developers:v3Dec 6, 2023

normanrz commented Dec 6, 2023
View reviewed changes

This was referenced Dec 7, 2023
Implement Variable Chunking (ZEP003) in V3 #1595
Closed
doc(v3): add v3 roadmap and design document #1583
Merged

jhamman mentioned this pull request Jan 24, 2024
Supporting Zarr-Python 3 Codec API zarr-developers/numcodecs#502
Closed

jhamman added this to the 3.0.0.alpha milestone Apr 22, 2024

sharkinsspatial mentioned this pull request Jul 24, 2024
Non-kerchunk backend for HDF5/netcdf4 files. zarr-developers/VirtualiZarr#87
Merged

dstansby mentioned this pull request Jan 26, 2025
Difference in behavior between 2.x and 3.x using identical compressor settings #2766
Closed

jhammanand others added 10 commits November 3, 2023 12:36

Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06
55b47b1
No code changes to Zarrita were made.

apply zarr lint rules
a31a4e5

zarrita -> v3
15f667c

v3/abc [wip]
c3f8764

use abcs plus implementation notes
0864187

working on making codecs extensible
78d0bc0

merge
8b86afa

adds index_location
4fac528

adds support for codec entry points
fcf79b6

adds tests from zarrita
9f97282

fixes types
f8bab5b

jhamman added the V3-compat label Dec 5, 2023

jhamman approved these changes Dec 6, 2023
View reviewed changes

normanrzand others added 2 commits December 6, 2023 11:34

Apply suggestions from code review
6c3e40a
Co-authored-by: Joe Hamman <jhamman1@gmail.com>

remove test codec from pyproject.toml
9749f25

jhamman merged commit 1a98886 into zarr-developers:v3Dec 6, 2023

normanrz commented Dec 6, 2023
View reviewed changes

This was referenced Dec 7, 2023
Implement Variable Chunking (ZEP003) in V3 #1595
Closed
doc(v3): add v3 roadmap and design document #1583
Merged

jhamman mentioned this pull request Jan 24, 2024
Supporting Zarr-Python 3 Codec API zarr-developers/numcodecs#502
Closed

jhamman added this to the 3.0.0.alpha milestone Apr 22, 2024

sharkinsspatial mentioned this pull request Jul 24, 2024
Non-kerchunk backend for HDF5/netcdf4 files. zarr-developers/VirtualiZarr#87
Merged

dstansby mentioned this pull request Jan 26, 2025
Difference in behavior between 2.x and 3.x using identical compressor settings #2766
Closed

jhammanand others added 10 commits November 3, 2023 12:36

Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06
55b47b1
No code changes to Zarrita were made.

apply zarr lint rules
a31a4e5

zarrita -> v3
15f667c

v3/abc [wip]
c3f8764

use abcs plus implementation notes
0864187

working on making codecs extensible
78d0bc0

merge
8b86afa

adds index_location
4fac528

adds support for codec entry points
fcf79b6

adds tests from zarrita
9f97282

fixes types
f8bab5b

jhamman added the V3-compat label Dec 5, 2023

jhamman approved these changes Dec 6, 2023
View reviewed changes

normanrzand others added 2 commits December 6, 2023 11:34

Apply suggestions from code review
6c3e40a
Co-authored-by: Joe Hamman <jhamman1@gmail.com>

remove test codec from pyproject.toml
9749f25

jhamman merged commit 1a98886 into zarr-developers:v3Dec 6, 2023

normanrz commented Dec 6, 2023
View reviewed changes

This was referenced Dec 7, 2023
Implement Variable Chunking (ZEP003) in V3 #1595
Closed
doc(v3): add v3 roadmap and design document #1583
Merged

jhamman mentioned this pull request Jan 24, 2024
Supporting Zarr-Python 3 Codec API zarr-developers/numcodecs#502
Closed

jhamman added this to the 3.0.0.alpha milestone Apr 22, 2024

sharkinsspatial mentioned this pull request Jul 24, 2024
Non-kerchunk backend for HDF5/netcdf4 files. zarr-developers/VirtualiZarr#87
Merged

dstansby mentioned this pull request Jan 26, 2025
Difference in behavior between 2.x and 3.x using identical compressor settings #2766
Closed

jhammanand others added 10 commits November 3, 2023 12:36

Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06
55b47b1
No code changes to Zarrita were made.

apply zarr lint rules
a31a4e5

zarrita -> v3
15f667c

v3/abc [wip]
c3f8764

use abcs plus implementation notes
0864187

working on making codecs extensible
78d0bc0

merge
8b86afa

adds index_location
4fac528

adds support for codec entry points
fcf79b6

adds tests from zarrita
9f97282

fixes types
f8bab5b

jhamman added the V3-compat label Dec 5, 2023

jhamman approved these changes Dec 6, 2023
View reviewed changes

normanrzand others added 2 commits December 6, 2023 11:34

Apply suggestions from code review
6c3e40a
Co-authored-by: Joe Hamman <jhamman1@gmail.com>

remove test codec from pyproject.toml
9749f25

jhamman merged commit 1a98886 into zarr-developers:v3Dec 6, 2023

normanrz commented Dec 6, 2023
View reviewed changes

This was referenced Dec 7, 2023
Implement Variable Chunking (ZEP003) in V3 #1595
Closed
doc(v3): add v3 roadmap and design document #1583
Merged

jhamman mentioned this pull request Jan 24, 2024
Supporting Zarr-Python 3 Codec API zarr-developers/numcodecs#502
Closed

jhamman added this to the 3.0.0.alpha milestone Apr 22, 2024

sharkinsspatial mentioned this pull request Jul 24, 2024
Non-kerchunk backend for HDF5/netcdf4 files. zarr-developers/VirtualiZarr#87
Merged

dstansby mentioned this pull request Jan 26, 2025
Difference in behavior between 2.x and 3.x using identical compressor settings #2766
Closed

jhammanand others added 10 commits November 3, 2023 12:36

Pull Zarrita into Zarr-Python @ 78274781ad64aef95772eb4b083f7ea9b7d03d06
55b47b1
No code changes to Zarrita were made.

apply zarr lint rules
a31a4e5

zarrita -> v3
15f667c

v3/abc [wip]
c3f8764

use abcs plus implementation notes
0864187

working on making codecs extensible
78d0bc0

merge
8b86afa

adds index_location
4fac528

adds support for codec entry points
fcf79b6

adds tests from zarrita
9f97282

fixes types
f8bab5b

jhamman added the V3-compat label Dec 5, 2023

jhamman approved these changes Dec 6, 2023
View reviewed changes

normanrzand others added 2 commits December 6, 2023 11:34

Apply suggestions from code review
6c3e40a
Co-authored-by: Joe Hamman <jhamman1@gmail.com>

remove test codec from pyproject.toml
9749f25

jhamman merged commit 1a98886 into zarr-developers:v3Dec 6, 2023

normanrz commented Dec 6, 2023
View reviewed changes

This was referenced Dec 7, 2023
Implement Variable Chunking (ZEP003) in V3 #1595
Closed
doc(v3): add v3 roadmap and design document #1583
Merged

jhamman mentioned this pull request Jan 24, 2024
Supporting Zarr-Python 3 Codec API zarr-developers/numcodecs#502
Closed

jhamman added this to the 3.0.0.alpha milestone Apr 22, 2024

sharkinsspatial mentioned this pull request Jul 24, 2024
Non-kerchunk backend for HDF5/netcdf4 files. zarr-developers/VirtualiZarr#87
Merged

dstansby mentioned this pull request Jan 26, 2025
Difference in behavior between 2.x and 3.x using identical compressor settings #2766
Closed

		@@ -59,24 +53,8 @@ def _structure_chunk_key_encoding_metadata(d: Dict[str, Any], _t) -> ChunkKeyEnc
		)

		def _structure_codec_metadata(d: Dict[str, Any], _t=None) -> CodecMetadata:

Uh oh!

Uh oh!

Conversation

normanrz commented Dec 5, 2023

Uh oh!

pep8speaks commented Dec 5, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comment last updated at 2023-12-06 10:51:06 UTC

Uh oh!

jhamman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

normanrz commented Dec 5, 2023

Uh oh!

pep8speaks commented Dec 5, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comment last updated at 2023-12-06 10:51:06 UTC

Uh oh!

jhamman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

normanrz commented Dec 5, 2023

Uh oh!

pep8speaks commented Dec 5, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comment last updated at 2023-12-06 10:51:06 UTC

Uh oh!

jhamman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

pep8speaks commented Dec 5, 2023•
edited
Loading

pep8speaks commented Dec 5, 2023•
edited
Loading

pep8speaks commented Dec 5, 2023•
edited
Loading

pep8speaks commented Dec 5, 2023•
edited
Loading