Revert change to default write_empty_chunks.

[vyasr]

This PR resolves#965 by changing the default of write_empty_chunks to True. I am not sure if we want to document the current limitations of write_empty_chunks=False, or if we want to treat those as bugs that will be fixed eventually. In the future we may want to error if a user sets write_empty_chunks=True with an unsupported data type, but that change should probably be made in a future PR that implements the solution described in #965 (comment) and #965 (comment)

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[jeromekelleher]

Great to see this, thanks @vyasr! cc @benjeffery@tomwhite

[joshmoore]

I assume the general consensus is to get this out as an immediate 2.11.2?

[jni]

That's my understanding!

[joshmoore]

Thanks, @vyasr!

Merging and I'll start preparing a v2 branch for releasing from. (As a side note: slightly surprised that none of the tests needed updating. I assume that means that none are testing the default value, eh?)

[jakirkham]

Thanks all! 😄

[joshmoore]

2.11.2 is released from the 2_11 branch with this PR backported.

[jakirkham]

Should we create a GitHub release for that tag as well?

[joshmoore]

Ah, apologies. I forgot that step. Yes, I'll do it now.

[jakirkham]

Not at all. Thanks for handling this Josh 🙏

[tomwhite]

Thanks for releasing 2.11.2 @joshmoore. Unfortunately, it looks like it doesn't fully revert the change:

>>>importzarr>>>zarr.__version__'2.11.2'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray']

Whereas with a previous version of Zarr (and the main branch):

>>>importzarr>>>zarr.__version__'2.10.3'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray', '0.0', '0.1']

The problem is in zarr.creation.create where the default is still write_empty_chunks=False:

zarr-python/zarr/creation.py

Line 23 in 04519bd

write_empty_chunks=False, **kwargs):

[jakirkham]

@tomwhite would you be willing to send a PR? 🙂

[tomwhite]

Opened #1005

[vyasr]

Ah sorry I guessed I missed one, sorry about that. Thanks for the patch @tomwhite!

[joshmoore]

@vyasr : not at all. My fault.

[joshmoore]

@d-v-b, if you are still planning to add an auto setting based on dtype, there are perhaps some other interesting comments regarding defaults in #8.

[vyasr]

This PR resolves#965 by changing the default of write_empty_chunks to True. I am not sure if we want to document the current limitations of write_empty_chunks=False, or if we want to treat those as bugs that will be fixed eventually. In the future we may want to error if a user sets write_empty_chunks=True with an unsupported data type, but that change should probably be made in a future PR that implements the solution described in #965 (comment) and #965 (comment)

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[jeromekelleher]

Great to see this, thanks @vyasr! cc @benjeffery@tomwhite

[joshmoore]

I assume the general consensus is to get this out as an immediate 2.11.2?

[jni]

That's my understanding!

[joshmoore]

Thanks, @vyasr!

Merging and I'll start preparing a v2 branch for releasing from. (As a side note: slightly surprised that none of the tests needed updating. I assume that means that none are testing the default value, eh?)

[jakirkham]

Thanks all! 😄

[joshmoore]

2.11.2 is released from the 2_11 branch with this PR backported.

[jakirkham]

Should we create a GitHub release for that tag as well?

[joshmoore]

Ah, apologies. I forgot that step. Yes, I'll do it now.

[jakirkham]

Not at all. Thanks for handling this Josh 🙏

[tomwhite]

Thanks for releasing 2.11.2 @joshmoore. Unfortunately, it looks like it doesn't fully revert the change:

>>>importzarr>>>zarr.__version__'2.11.2'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray']

Whereas with a previous version of Zarr (and the main branch):

>>>importzarr>>>zarr.__version__'2.10.3'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray', '0.0', '0.1']

The problem is in zarr.creation.create where the default is still write_empty_chunks=False:

zarr-python/zarr/creation.py

Line 23 in 04519bd

write_empty_chunks=False, **kwargs):

[jakirkham]

@tomwhite would you be willing to send a PR? 🙂

[tomwhite]

Opened #1005

[vyasr]

Ah sorry I guessed I missed one, sorry about that. Thanks for the patch @tomwhite!

[joshmoore]

@vyasr : not at all. My fault.

[joshmoore]

@d-v-b, if you are still planning to add an auto setting based on dtype, there are perhaps some other interesting comments regarding defaults in #8.

[vyasr]

This PR resolves#965 by changing the default of write_empty_chunks to True. I am not sure if we want to document the current limitations of write_empty_chunks=False, or if we want to treat those as bugs that will be fixed eventually. In the future we may want to error if a user sets write_empty_chunks=True with an unsupported data type, but that change should probably be made in a future PR that implements the solution described in #965 (comment) and #965 (comment)

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[jeromekelleher]

Great to see this, thanks @vyasr! cc @benjeffery@tomwhite

[joshmoore]

I assume the general consensus is to get this out as an immediate 2.11.2?

[jni]

That's my understanding!

[joshmoore]

Thanks, @vyasr!

Merging and I'll start preparing a v2 branch for releasing from. (As a side note: slightly surprised that none of the tests needed updating. I assume that means that none are testing the default value, eh?)

[jakirkham]

Thanks all! 😄

[joshmoore]

2.11.2 is released from the 2_11 branch with this PR backported.

[jakirkham]

Should we create a GitHub release for that tag as well?

[joshmoore]

Ah, apologies. I forgot that step. Yes, I'll do it now.

[jakirkham]

Not at all. Thanks for handling this Josh 🙏

[tomwhite]

Thanks for releasing 2.11.2 @joshmoore. Unfortunately, it looks like it doesn't fully revert the change:

>>>importzarr>>>zarr.__version__'2.11.2'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray']

Whereas with a previous version of Zarr (and the main branch):

>>>importzarr>>>zarr.__version__'2.10.3'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray', '0.0', '0.1']

The problem is in zarr.creation.create where the default is still write_empty_chunks=False:

zarr-python/zarr/creation.py

Line 23 in 04519bd

write_empty_chunks=False, **kwargs):

[jakirkham]

@tomwhite would you be willing to send a PR? 🙂

[tomwhite]

Opened #1005

[vyasr]

Ah sorry I guessed I missed one, sorry about that. Thanks for the patch @tomwhite!

[joshmoore]

@vyasr : not at all. My fault.

[joshmoore]

@d-v-b, if you are still planning to add an auto setting based on dtype, there are perhaps some other interesting comments regarding defaults in #8.

[vyasr]

This PR resolves#965 by changing the default of write_empty_chunks to True. I am not sure if we want to document the current limitations of write_empty_chunks=False, or if we want to treat those as bugs that will be fixed eventually. In the future we may want to error if a user sets write_empty_chunks=True with an unsupported data type, but that change should probably be made in a future PR that implements the solution described in #965 (comment) and #965 (comment)

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[jeromekelleher]

Great to see this, thanks @vyasr! cc @benjeffery@tomwhite

[joshmoore]

I assume the general consensus is to get this out as an immediate 2.11.2?

[jni]

That's my understanding!

[joshmoore]

Thanks, @vyasr!

Merging and I'll start preparing a v2 branch for releasing from. (As a side note: slightly surprised that none of the tests needed updating. I assume that means that none are testing the default value, eh?)

[jakirkham]

Thanks all! 😄

[joshmoore]

2.11.2 is released from the 2_11 branch with this PR backported.

[jakirkham]

Should we create a GitHub release for that tag as well?

[joshmoore]

Ah, apologies. I forgot that step. Yes, I'll do it now.

[jakirkham]

Not at all. Thanks for handling this Josh 🙏

[tomwhite]

Thanks for releasing 2.11.2 @joshmoore. Unfortunately, it looks like it doesn't fully revert the change:

>>>importzarr>>>zarr.__version__'2.11.2'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray']

Whereas with a previous version of Zarr (and the main branch):

>>>importzarr>>>zarr.__version__'2.10.3'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray', '0.0', '0.1']

The problem is in zarr.creation.create where the default is still write_empty_chunks=False:

zarr-python/zarr/creation.py

Line 23 in 04519bd

write_empty_chunks=False, **kwargs):

[jakirkham]

@tomwhite would you be willing to send a PR? 🙂

[tomwhite]

Opened #1005

[vyasr]

Ah sorry I guessed I missed one, sorry about that. Thanks for the patch @tomwhite!

[joshmoore]

@vyasr : not at all. My fault.

[joshmoore]

@d-v-b, if you are still planning to add an auto setting based on dtype, there are perhaps some other interesting comments regarding defaults in #8.

[vyasr]

This PR resolves#965 by changing the default of write_empty_chunks to True. I am not sure if we want to document the current limitations of write_empty_chunks=False, or if we want to treat those as bugs that will be fixed eventually. In the future we may want to error if a user sets write_empty_chunks=True with an unsupported data type, but that change should probably be made in a future PR that implements the solution described in #965 (comment) and #965 (comment)

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[jeromekelleher]

Great to see this, thanks @vyasr! cc @benjeffery@tomwhite

[joshmoore]

I assume the general consensus is to get this out as an immediate 2.11.2?

[jni]

That's my understanding!

[joshmoore]

Thanks, @vyasr!

Merging and I'll start preparing a v2 branch for releasing from. (As a side note: slightly surprised that none of the tests needed updating. I assume that means that none are testing the default value, eh?)

[jakirkham]

Thanks all! 😄

[joshmoore]

2.11.2 is released from the 2_11 branch with this PR backported.

[jakirkham]

Should we create a GitHub release for that tag as well?

[joshmoore]

Ah, apologies. I forgot that step. Yes, I'll do it now.

[jakirkham]

Not at all. Thanks for handling this Josh 🙏

[tomwhite]

Thanks for releasing 2.11.2 @joshmoore. Unfortunately, it looks like it doesn't fully revert the change:

>>>importzarr>>>zarr.__version__'2.11.2'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray']

Whereas with a previous version of Zarr (and the main branch):

>>>importzarr>>>zarr.__version__'2.10.3'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray', '0.0', '0.1']

The problem is in zarr.creation.create where the default is still write_empty_chunks=False:

zarr-python/zarr/creation.py

Line 23 in 04519bd

write_empty_chunks=False, **kwargs):

[jakirkham]

@tomwhite would you be willing to send a PR? 🙂

[tomwhite]

Opened #1005

[vyasr]

Ah sorry I guessed I missed one, sorry about that. Thanks for the patch @tomwhite!

[joshmoore]

@vyasr : not at all. My fault.

[joshmoore]

@d-v-b, if you are still planning to add an auto setting based on dtype, there are perhaps some other interesting comments regarding defaults in #8.

[vyasr]

This PR resolves#965 by changing the default of write_empty_chunks to True. I am not sure if we want to document the current limitations of write_empty_chunks=False, or if we want to treat those as bugs that will be fixed eventually. In the future we may want to error if a user sets write_empty_chunks=True with an unsupported data type, but that change should probably be made in a future PR that implements the solution described in #965 (comment) and #965 (comment)

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[jeromekelleher]

Great to see this, thanks @vyasr! cc @benjeffery@tomwhite

[joshmoore]

I assume the general consensus is to get this out as an immediate 2.11.2?

[jni]

That's my understanding!

[joshmoore]

Thanks, @vyasr!

Merging and I'll start preparing a v2 branch for releasing from. (As a side note: slightly surprised that none of the tests needed updating. I assume that means that none are testing the default value, eh?)

[jakirkham]

Thanks all! 😄

[joshmoore]

2.11.2 is released from the 2_11 branch with this PR backported.

[jakirkham]

Should we create a GitHub release for that tag as well?

[joshmoore]

Ah, apologies. I forgot that step. Yes, I'll do it now.

[jakirkham]

Not at all. Thanks for handling this Josh 🙏

[tomwhite]

Thanks for releasing 2.11.2 @joshmoore. Unfortunately, it looks like it doesn't fully revert the change:

>>>importzarr>>>zarr.__version__'2.11.2'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray']

Whereas with a previous version of Zarr (and the main branch):

>>>importzarr>>>zarr.__version__'2.10.3'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray', '0.0', '0.1']

The problem is in zarr.creation.create where the default is still write_empty_chunks=False:

zarr-python/zarr/creation.py

Line 23 in 04519bd

write_empty_chunks=False, **kwargs):

[jakirkham]

@tomwhite would you be willing to send a PR? 🙂

[tomwhite]

Opened #1005

[vyasr]

Ah sorry I guessed I missed one, sorry about that. Thanks for the patch @tomwhite!

[joshmoore]

@vyasr : not at all. My fault.

[joshmoore]

@d-v-b, if you are still planning to add an auto setting based on dtype, there are perhaps some other interesting comments regarding defaults in #8.

[vyasr]

This PR resolves#965 by changing the default of write_empty_chunks to True. I am not sure if we want to document the current limitations of write_empty_chunks=False, or if we want to treat those as bugs that will be fixed eventually. In the future we may want to error if a user sets write_empty_chunks=True with an unsupported data type, but that change should probably be made in a future PR that implements the solution described in #965 (comment) and #965 (comment)

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[jeromekelleher]

Great to see this, thanks @vyasr! cc @benjeffery@tomwhite

[joshmoore]

I assume the general consensus is to get this out as an immediate 2.11.2?

[jni]

That's my understanding!

[joshmoore]

Thanks, @vyasr!

Merging and I'll start preparing a v2 branch for releasing from. (As a side note: slightly surprised that none of the tests needed updating. I assume that means that none are testing the default value, eh?)

[jakirkham]

Thanks all! 😄

[joshmoore]

2.11.2 is released from the 2_11 branch with this PR backported.

[jakirkham]

Should we create a GitHub release for that tag as well?

[joshmoore]

Ah, apologies. I forgot that step. Yes, I'll do it now.

[jakirkham]

Not at all. Thanks for handling this Josh 🙏

[tomwhite]

Thanks for releasing 2.11.2 @joshmoore. Unfortunately, it looks like it doesn't fully revert the change:

>>>importzarr>>>zarr.__version__'2.11.2'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray']

Whereas with a previous version of Zarr (and the main branch):

>>>importzarr>>>zarr.__version__'2.10.3'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray', '0.0', '0.1']

The problem is in zarr.creation.create where the default is still write_empty_chunks=False:

zarr-python/zarr/creation.py

Line 23 in 04519bd

write_empty_chunks=False, **kwargs):

[jakirkham]

@tomwhite would you be willing to send a PR? 🙂

[tomwhite]

Opened #1005

[vyasr]

Ah sorry I guessed I missed one, sorry about that. Thanks for the patch @tomwhite!

[joshmoore]

@vyasr : not at all. My fault.

[joshmoore]

@d-v-b, if you are still planning to add an auto setting based on dtype, there are perhaps some other interesting comments regarding defaults in #8.

[vyasr]

This PR resolves#965 by changing the default of write_empty_chunks to True. I am not sure if we want to document the current limitations of write_empty_chunks=False, or if we want to treat those as bugs that will be fixed eventually. In the future we may want to error if a user sets write_empty_chunks=True with an unsupported data type, but that change should probably be made in a future PR that implements the solution described in #965 (comment) and #965 (comment)

TODO:

• Add unit tests and/or doctests in docstrings
• Add docstrings and API docs for any new/modified user-facing classes and functions
• New/modified features documented in docs/tutorial.rst
• Changes documented in docs/release.rst
• GitHub Actions have all passed
• Test coverage is 100% (Codecov passes)

[jeromekelleher]

Great to see this, thanks @vyasr! cc @benjeffery@tomwhite

[joshmoore]

I assume the general consensus is to get this out as an immediate 2.11.2?

[jni]

That's my understanding!

[joshmoore]

Thanks, @vyasr!

Merging and I'll start preparing a v2 branch for releasing from. (As a side note: slightly surprised that none of the tests needed updating. I assume that means that none are testing the default value, eh?)

[jakirkham]

Thanks all! 😄

[joshmoore]

2.11.2 is released from the 2_11 branch with this PR backported.

[jakirkham]

Should we create a GitHub release for that tag as well?

[joshmoore]

Ah, apologies. I forgot that step. Yes, I'll do it now.

[jakirkham]

Not at all. Thanks for handling this Josh 🙏

[tomwhite]

Thanks for releasing 2.11.2 @joshmoore. Unfortunately, it looks like it doesn't fully revert the change:

>>>importzarr>>>zarr.__version__'2.11.2'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray']

Whereas with a previous version of Zarr (and the main branch):

>>>importzarr>>>zarr.__version__'2.10.3'>>>a=zarr.create((100, 100), chunks=(100, 50), dtype="i4", store="example.zarr")
>>>a[:] =0>>>importos>>>os.listdir("example.zarr")
['.zarray', '0.0', '0.1']

The problem is in zarr.creation.create where the default is still write_empty_chunks=False:

zarr-python/zarr/creation.py

Line 23 in 04519bd

write_empty_chunks=False, **kwargs):

[jakirkham]

@tomwhite would you be willing to send a PR? 🙂

[tomwhite]

Opened #1005

[vyasr]

Ah sorry I guessed I missed one, sorry about that. Thanks for the patch @tomwhite!

[joshmoore]

@vyasr : not at all. My fault.

[joshmoore]

@d-v-b, if you are still planning to add an auto setting based on dtype, there are perhaps some other interesting comments regarding defaults in #8.