List group members level by level instead of enumerating all keys

[konstibob]

Group.list() called storeHandle.list(), which is fully recursive and returns every key below the group, including all chunk keys. Listing a group with 1M chunks on S3 cost ~1000 paginated requests to find a handful of zarr.json files.

Walk the hierarchy with listChildren() instead, one level at a time, and never descend into arrays. The implementation moves to core.Group, so v2 and v3 share it, and adds members() for the immediate children only.

Also:

• S3Store.listChildren() used a single listObjectsV2 call, silently truncating groups with more than 1000 children. Use the paginator.
• FilesystemStore.get() threw instead of returning null when a path component is a file rather than a directory. Probing a plain file next to a group's nodes is normal now, and such a key holds no data.

[konstibob]

Group.list() called storeHandle.list(), which is fully recursive and returns every key below the group, including all chunk keys. Listing a group with 1M chunks on S3 cost ~1000 paginated requests to find a handful of zarr.json files.

Walk the hierarchy with listChildren() instead, one level at a time, and never descend into arrays. The implementation moves to core.Group, so v2 and v3 share it, and adds members() for the immediate children only.

Also:

• S3Store.listChildren() used a single listObjectsV2 call, silently truncating groups with more than 1000 children. Use the paginator.
• FilesystemStore.get() threw instead of returning null when a path component is a file rather than a directory. Probing a plain file next to a group's nodes is normal now, and such a key holds no data.

[konstibob]

Group.list() called storeHandle.list(), which is fully recursive and returns every key below the group, including all chunk keys. Listing a group with 1M chunks on S3 cost ~1000 paginated requests to find a handful of zarr.json files.

Walk the hierarchy with listChildren() instead, one level at a time, and never descend into arrays. The implementation moves to core.Group, so v2 and v3 share it, and adds members() for the immediate children only.

Also:

• S3Store.listChildren() used a single listObjectsV2 call, silently truncating groups with more than 1000 children. Use the paginator.
• FilesystemStore.get() threw instead of returning null when a path component is a file rather than a directory. Probing a plain file next to a group's nodes is normal now, and such a key holds no data.

[konstibob]

Group.list() called storeHandle.list(), which is fully recursive and returns every key below the group, including all chunk keys. Listing a group with 1M chunks on S3 cost ~1000 paginated requests to find a handful of zarr.json files.

Walk the hierarchy with listChildren() instead, one level at a time, and never descend into arrays. The implementation moves to core.Group, so v2 and v3 share it, and adds members() for the immediate children only.

Also:

• S3Store.listChildren() used a single listObjectsV2 call, silently truncating groups with more than 1000 children. Use the paginator.
• FilesystemStore.get() threw instead of returning null when a path component is a file rather than a directory. Probing a plain file next to a group's nodes is normal now, and such a key holds no data.

[konstibob]

Group.list() called storeHandle.list(), which is fully recursive and returns every key below the group, including all chunk keys. Listing a group with 1M chunks on S3 cost ~1000 paginated requests to find a handful of zarr.json files.

Walk the hierarchy with listChildren() instead, one level at a time, and never descend into arrays. The implementation moves to core.Group, so v2 and v3 share it, and adds members() for the immediate children only.

Also:

• S3Store.listChildren() used a single listObjectsV2 call, silently truncating groups with more than 1000 children. Use the paginator.
• FilesystemStore.get() threw instead of returning null when a path component is a file rather than a directory. Probing a plain file next to a group's nodes is normal now, and such a key holds no data.

[konstibob]

Group.list() called storeHandle.list(), which is fully recursive and returns every key below the group, including all chunk keys. Listing a group with 1M chunks on S3 cost ~1000 paginated requests to find a handful of zarr.json files.

Walk the hierarchy with listChildren() instead, one level at a time, and never descend into arrays. The implementation moves to core.Group, so v2 and v3 share it, and adds members() for the immediate children only.

Also:

• S3Store.listChildren() used a single listObjectsV2 call, silently truncating groups with more than 1000 children. Use the paginator.
• FilesystemStore.get() threw instead of returning null when a path component is a file rather than a directory. Probing a plain file next to a group's nodes is normal now, and such a key holds no data.

[konstibob]

Group.list() called storeHandle.list(), which is fully recursive and returns every key below the group, including all chunk keys. Listing a group with 1M chunks on S3 cost ~1000 paginated requests to find a handful of zarr.json files.

Walk the hierarchy with listChildren() instead, one level at a time, and never descend into arrays. The implementation moves to core.Group, so v2 and v3 share it, and adds members() for the immediate children only.

Also:

• S3Store.listChildren() used a single listObjectsV2 call, silently truncating groups with more than 1000 children. Use the paginator.
• FilesystemStore.get() threw instead of returning null when a path component is a file rather than a directory. Probing a plain file next to a group's nodes is normal now, and such a key holds no data.

[konstibob]

Group.list() called storeHandle.list(), which is fully recursive and returns every key below the group, including all chunk keys. Listing a group with 1M chunks on S3 cost ~1000 paginated requests to find a handful of zarr.json files.

Walk the hierarchy with listChildren() instead, one level at a time, and never descend into arrays. The implementation moves to core.Group, so v2 and v3 share it, and adds members() for the immediate children only.

Also:

• S3Store.listChildren() used a single listObjectsV2 call, silently truncating groups with more than 1000 children. Use the paginator.
• FilesystemStore.get() threw instead of returning null when a path component is a file rather than a directory. Probing a plain file next to a group's nodes is normal now, and such a key holds no data.