[DRAFT] V3 spec implmentation.

[Carreau]

This is mostly opened to foster discussion and need code from https://github.com/Carreau/zarr-spec-v3-impl.

IN the above mentioned repository I'm working on looking at what an
implementation of the spec v3 could look to inform us on the possible
transition and compatiblity shims.

So far I have a rough implementation of an in memory v3 store as well as
multiple utilities.

1. A base class that automatically provide sync version of all async
   method of a class. I'm playing with the idea of having most method
   async as this may be useful in some context.

For example when creating a array at /a/b/c/d/e/f/g/h/i/j/k you want
to check that None of the parents are arrays, which can be done with N
async requests.

2. An adapted class that wraps a v3 store and provide a v2 API.
   My though is that most code is currently v2 compatible and it would be
   useful for legacy codebase and early testing of store.

3. a class that wrap 2 stores, a reference and a tested one, replicate
   operation on both stores, and abort if it sees any difference in
   behavior. This could help to catch changes in behavior.

The tests in this PR start to test the v3 memorystore and compare it to
the v2 memorystore.

[pep8speaks]

Hello @Carreau! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file zarr/v3/__init__.py:

Line 26:1: E402 module level import not at top of file
Line 218:75: E203 whitespace before ':'

Comment last updated at 2020-08-26 16:50:57 UTC

[Carreau]

Question on v3 implementation route. Do you want a complete rewrite in a separate module ? Or do you want to still use the current Group/Array class as well as the utilities functions in zarr.storage / zarr.meta ...etc.

Many of these function will need to be mave "version aware", some of them might be able to get the current version as they are given a store. But other (decode/encode_array_metadata) are not and should be likely given a kwarg.

[Carreau]

Pr was at 96732fc97e86fdb07f84d8b307a11d1f311a0fc3 , will rebase /squash and force-push.

[joshmoore]

This has been completed by #898 (plus follow ons like #1007)

[Carreau]

This is mostly opened to foster discussion and need code from https://github.com/Carreau/zarr-spec-v3-impl.

IN the above mentioned repository I'm working on looking at what an
implementation of the spec v3 could look to inform us on the possible
transition and compatiblity shims.

So far I have a rough implementation of an in memory v3 store as well as
multiple utilities.

1. A base class that automatically provide sync version of all async
   method of a class. I'm playing with the idea of having most method
   async as this may be useful in some context.

For example when creating a array at /a/b/c/d/e/f/g/h/i/j/k you want
to check that None of the parents are arrays, which can be done with N
async requests.

2. An adapted class that wraps a v3 store and provide a v2 API.
   My though is that most code is currently v2 compatible and it would be
   useful for legacy codebase and early testing of store.

3. a class that wrap 2 stores, a reference and a tested one, replicate
   operation on both stores, and abort if it sees any difference in
   behavior. This could help to catch changes in behavior.

The tests in this PR start to test the v3 memorystore and compare it to
the v2 memorystore.

[pep8speaks]

Hello @Carreau! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file zarr/v3/__init__.py:

Line 26:1: E402 module level import not at top of file
Line 218:75: E203 whitespace before ':'

Comment last updated at 2020-08-26 16:50:57 UTC

[Carreau]

Question on v3 implementation route. Do you want a complete rewrite in a separate module ? Or do you want to still use the current Group/Array class as well as the utilities functions in zarr.storage / zarr.meta ...etc.

Many of these function will need to be mave "version aware", some of them might be able to get the current version as they are given a store. But other (decode/encode_array_metadata) are not and should be likely given a kwarg.

[Carreau]

Pr was at 96732fc97e86fdb07f84d8b307a11d1f311a0fc3 , will rebase /squash and force-push.

[joshmoore]

This has been completed by #898 (plus follow ons like #1007)

[Carreau]

This is mostly opened to foster discussion and need code from https://github.com/Carreau/zarr-spec-v3-impl.

IN the above mentioned repository I'm working on looking at what an
implementation of the spec v3 could look to inform us on the possible
transition and compatiblity shims.

So far I have a rough implementation of an in memory v3 store as well as
multiple utilities.

1. A base class that automatically provide sync version of all async
   method of a class. I'm playing with the idea of having most method
   async as this may be useful in some context.

For example when creating a array at /a/b/c/d/e/f/g/h/i/j/k you want
to check that None of the parents are arrays, which can be done with N
async requests.

2. An adapted class that wraps a v3 store and provide a v2 API.
   My though is that most code is currently v2 compatible and it would be
   useful for legacy codebase and early testing of store.

3. a class that wrap 2 stores, a reference and a tested one, replicate
   operation on both stores, and abort if it sees any difference in
   behavior. This could help to catch changes in behavior.

The tests in this PR start to test the v3 memorystore and compare it to
the v2 memorystore.

[pep8speaks]

Hello @Carreau! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file zarr/v3/__init__.py:

Line 26:1: E402 module level import not at top of file
Line 218:75: E203 whitespace before ':'

Comment last updated at 2020-08-26 16:50:57 UTC

[Carreau]

Question on v3 implementation route. Do you want a complete rewrite in a separate module ? Or do you want to still use the current Group/Array class as well as the utilities functions in zarr.storage / zarr.meta ...etc.

Many of these function will need to be mave "version aware", some of them might be able to get the current version as they are given a store. But other (decode/encode_array_metadata) are not and should be likely given a kwarg.

[Carreau]

Pr was at 96732fc97e86fdb07f84d8b307a11d1f311a0fc3 , will rebase /squash and force-push.

[joshmoore]

This has been completed by #898 (plus follow ons like #1007)

[Carreau]

This is mostly opened to foster discussion and need code from https://github.com/Carreau/zarr-spec-v3-impl.

IN the above mentioned repository I'm working on looking at what an
implementation of the spec v3 could look to inform us on the possible
transition and compatiblity shims.

So far I have a rough implementation of an in memory v3 store as well as
multiple utilities.

1. A base class that automatically provide sync version of all async
   method of a class. I'm playing with the idea of having most method
   async as this may be useful in some context.

For example when creating a array at /a/b/c/d/e/f/g/h/i/j/k you want
to check that None of the parents are arrays, which can be done with N
async requests.

2. An adapted class that wraps a v3 store and provide a v2 API.
   My though is that most code is currently v2 compatible and it would be
   useful for legacy codebase and early testing of store.

3. a class that wrap 2 stores, a reference and a tested one, replicate
   operation on both stores, and abort if it sees any difference in
   behavior. This could help to catch changes in behavior.

The tests in this PR start to test the v3 memorystore and compare it to
the v2 memorystore.

[pep8speaks]

Hello @Carreau! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file zarr/v3/__init__.py:

Line 26:1: E402 module level import not at top of file
Line 218:75: E203 whitespace before ':'

Comment last updated at 2020-08-26 16:50:57 UTC

[Carreau]

Question on v3 implementation route. Do you want a complete rewrite in a separate module ? Or do you want to still use the current Group/Array class as well as the utilities functions in zarr.storage / zarr.meta ...etc.

Many of these function will need to be mave "version aware", some of them might be able to get the current version as they are given a store. But other (decode/encode_array_metadata) are not and should be likely given a kwarg.

[Carreau]

Pr was at 96732fc97e86fdb07f84d8b307a11d1f311a0fc3 , will rebase /squash and force-push.

[joshmoore]

This has been completed by #898 (plus follow ons like #1007)

[Carreau]

This is mostly opened to foster discussion and need code from https://github.com/Carreau/zarr-spec-v3-impl.

IN the above mentioned repository I'm working on looking at what an
implementation of the spec v3 could look to inform us on the possible
transition and compatiblity shims.

So far I have a rough implementation of an in memory v3 store as well as
multiple utilities.

1. A base class that automatically provide sync version of all async
   method of a class. I'm playing with the idea of having most method
   async as this may be useful in some context.

For example when creating a array at /a/b/c/d/e/f/g/h/i/j/k you want
to check that None of the parents are arrays, which can be done with N
async requests.

2. An adapted class that wraps a v3 store and provide a v2 API.
   My though is that most code is currently v2 compatible and it would be
   useful for legacy codebase and early testing of store.

3. a class that wrap 2 stores, a reference and a tested one, replicate
   operation on both stores, and abort if it sees any difference in
   behavior. This could help to catch changes in behavior.

The tests in this PR start to test the v3 memorystore and compare it to
the v2 memorystore.

[pep8speaks]

Hello @Carreau! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file zarr/v3/__init__.py:

Line 26:1: E402 module level import not at top of file
Line 218:75: E203 whitespace before ':'

Comment last updated at 2020-08-26 16:50:57 UTC

[Carreau]

Question on v3 implementation route. Do you want a complete rewrite in a separate module ? Or do you want to still use the current Group/Array class as well as the utilities functions in zarr.storage / zarr.meta ...etc.

Many of these function will need to be mave "version aware", some of them might be able to get the current version as they are given a store. But other (decode/encode_array_metadata) are not and should be likely given a kwarg.

[Carreau]

Pr was at 96732fc97e86fdb07f84d8b307a11d1f311a0fc3 , will rebase /squash and force-push.

[joshmoore]

This has been completed by #898 (plus follow ons like #1007)

[Carreau]

This is mostly opened to foster discussion and need code from https://github.com/Carreau/zarr-spec-v3-impl.

IN the above mentioned repository I'm working on looking at what an
implementation of the spec v3 could look to inform us on the possible
transition and compatiblity shims.

So far I have a rough implementation of an in memory v3 store as well as
multiple utilities.

1. A base class that automatically provide sync version of all async
   method of a class. I'm playing with the idea of having most method
   async as this may be useful in some context.

For example when creating a array at /a/b/c/d/e/f/g/h/i/j/k you want
to check that None of the parents are arrays, which can be done with N
async requests.

2. An adapted class that wraps a v3 store and provide a v2 API.
   My though is that most code is currently v2 compatible and it would be
   useful for legacy codebase and early testing of store.

3. a class that wrap 2 stores, a reference and a tested one, replicate
   operation on both stores, and abort if it sees any difference in
   behavior. This could help to catch changes in behavior.

The tests in this PR start to test the v3 memorystore and compare it to
the v2 memorystore.

[pep8speaks]

Hello @Carreau! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file zarr/v3/__init__.py:

Line 26:1: E402 module level import not at top of file
Line 218:75: E203 whitespace before ':'

Comment last updated at 2020-08-26 16:50:57 UTC

[Carreau]

Question on v3 implementation route. Do you want a complete rewrite in a separate module ? Or do you want to still use the current Group/Array class as well as the utilities functions in zarr.storage / zarr.meta ...etc.

Many of these function will need to be mave "version aware", some of them might be able to get the current version as they are given a store. But other (decode/encode_array_metadata) are not and should be likely given a kwarg.

[Carreau]

Pr was at 96732fc97e86fdb07f84d8b307a11d1f311a0fc3 , will rebase /squash and force-push.

[joshmoore]

This has been completed by #898 (plus follow ons like #1007)

[Carreau]

This is mostly opened to foster discussion and need code from https://github.com/Carreau/zarr-spec-v3-impl.

IN the above mentioned repository I'm working on looking at what an
implementation of the spec v3 could look to inform us on the possible
transition and compatiblity shims.

So far I have a rough implementation of an in memory v3 store as well as
multiple utilities.

1. A base class that automatically provide sync version of all async
   method of a class. I'm playing with the idea of having most method
   async as this may be useful in some context.

For example when creating a array at /a/b/c/d/e/f/g/h/i/j/k you want
to check that None of the parents are arrays, which can be done with N
async requests.

2. An adapted class that wraps a v3 store and provide a v2 API.
   My though is that most code is currently v2 compatible and it would be
   useful for legacy codebase and early testing of store.

3. a class that wrap 2 stores, a reference and a tested one, replicate
   operation on both stores, and abort if it sees any difference in
   behavior. This could help to catch changes in behavior.

The tests in this PR start to test the v3 memorystore and compare it to
the v2 memorystore.

[pep8speaks]

Hello @Carreau! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file zarr/v3/__init__.py:

Line 26:1: E402 module level import not at top of file
Line 218:75: E203 whitespace before ':'

Comment last updated at 2020-08-26 16:50:57 UTC

[Carreau]

Question on v3 implementation route. Do you want a complete rewrite in a separate module ? Or do you want to still use the current Group/Array class as well as the utilities functions in zarr.storage / zarr.meta ...etc.

Many of these function will need to be mave "version aware", some of them might be able to get the current version as they are given a store. But other (decode/encode_array_metadata) are not and should be likely given a kwarg.

[Carreau]

Pr was at 96732fc97e86fdb07f84d8b307a11d1f311a0fc3 , will rebase /squash and force-push.

[joshmoore]

This has been completed by #898 (plus follow ons like #1007)

[Carreau]

This is mostly opened to foster discussion and need code from https://github.com/Carreau/zarr-spec-v3-impl.

IN the above mentioned repository I'm working on looking at what an
implementation of the spec v3 could look to inform us on the possible
transition and compatiblity shims.

So far I have a rough implementation of an in memory v3 store as well as
multiple utilities.

1. A base class that automatically provide sync version of all async
   method of a class. I'm playing with the idea of having most method
   async as this may be useful in some context.

For example when creating a array at /a/b/c/d/e/f/g/h/i/j/k you want
to check that None of the parents are arrays, which can be done with N
async requests.

2. An adapted class that wraps a v3 store and provide a v2 API.
   My though is that most code is currently v2 compatible and it would be
   useful for legacy codebase and early testing of store.

3. a class that wrap 2 stores, a reference and a tested one, replicate
   operation on both stores, and abort if it sees any difference in
   behavior. This could help to catch changes in behavior.

The tests in this PR start to test the v3 memorystore and compare it to
the v2 memorystore.

[pep8speaks]

Hello @Carreau! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file zarr/v3/__init__.py:

Line 26:1: E402 module level import not at top of file
Line 218:75: E203 whitespace before ':'

Comment last updated at 2020-08-26 16:50:57 UTC

[Carreau]

Question on v3 implementation route. Do you want a complete rewrite in a separate module ? Or do you want to still use the current Group/Array class as well as the utilities functions in zarr.storage / zarr.meta ...etc.

Many of these function will need to be mave "version aware", some of them might be able to get the current version as they are given a store. But other (decode/encode_array_metadata) are not and should be likely given a kwarg.

[Carreau]

Pr was at 96732fc97e86fdb07f84d8b307a11d1f311a0fc3 , will rebase /squash and force-push.

[joshmoore]

This has been completed by #898 (plus follow ons like #1007)