bug in v2 codec pipeline

[dcherian]

Zarr version

main

Description

There is a bug in these lines:

cdata is not of type bytes in the else clause:

zarr-python/src/zarr/codecs/_v2.py

Lines 80 to 86 in eb85ee7

	# compress
	if self.compressor:
	cdata = await asyncio.to_thread(self.compressor.encode, chunk)
	else:
	cdata = chunk
	return chunk_spec.prototype.buffer.from_bytes(cdata)

cc @normanrz

Steps to reproduce

This is triggering failures in xarray's test suite. I haven't succeeded at making a minimal reproducer.

Additional output

No response

[dcherian]

Ah this occurs when self.filters is None and self.compressor is None

[normanrz]

I just tried to reproduce this with

>>> import zarr
>>> import numpy as np
>>> a = zarr.Array.create("test.zarr", shape=(12,12,12), chunks=(12,12,12), dtype="uint8", zarr_format=2)
>>> a
<Array file://test.zarr shape=(12, 12, 12) dtype=uint8>
>>> str(a.metadata.filters)
'None'
>>> str(a.metadata.compressor)
'None'
>>> a[:] = np.ones((12,12,12))
>>> a[:]
array([[[1, 1, 1, ..., 1, 1, 1], ...

This seems to have worked.

[dcherian]

Here you go . When you set using a slice of a different array, it's not C-contiguous

import numpy as np
import zarr
from zarr.storage import MemoryStore

a = zarr.Array.create(MemoryStore({}), shape=(10,8), chunks=(3,3), fill_value=np.nan, dtype="float64", zarr_format=2, exists_ok=True)
print(a.metadata)
array = np.ones(a.shape)
a[slice(6, 9, None), slice(3, 6, None)] = array[slice(6, 9, None), slice(3, 6, None)] # The slice on the RHS is important
# ValueError: ndarray is not C-contiguous