Bump fsspec from 2021.6.0 to 2021.6.1 - #2
Closed
dependabot[bot] wants to merge 1 commit into
Closed
Conversation
Bumps [fsspec](https://github.com/intake/filesystem_spec) from 2021.6.0 to 2021.6.1. - [Release notes](https://github.com/intake/filesystem_spec/releases) - [Commits](fsspec/filesystem_spec@2021.06.0...2021.06.1) --- updated-dependencies: - dependency-name: fsspec dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Author
Superseded by #5. |
d-v-b added a commit
that referenced
this pull request
May 7, 2026
The function bridged NumPy-style negative indices to TensorStore-style absolute coordinates. It was originally used by the eager-path rewiring in PR zarr-developers#3906 (4 callsites in src/zarr/core/array.py). This PR has no such rewiring — PR #2's lazy view materializes via the existing eager indexing path, which already handles negatives — so the helper has no callers in our shipping plan. Re-add when (and if) the internal-rewiring follow-up arrives. Removing it now keeps the package surface lean and the diff focused.
d-v-b added a commit
that referenced
this pull request
Jul 31, 2026
`arr[::-1]` reverses. One desugaring rule covers both signs, as TensorStore 0.1.84 does and as ndsel PR #2 now specifies: omitted bounds resolve on the side the traversal starts and stops (`hi-1` and `lo-1` going down), the source interval is [start, stop) going up and [stop+1, start+1) going down, an empty interval is legal at any coordinate, an interval running the wrong way is an error rather than a silent empty, and the origin is trunc(start/step) for either sign. The corpus is re-vendored from ndsel 92d6a32 in this same commit, because it is the definition of correct here: `slice.json` gains ten negative-step fixtures, `errors.json` retires `negative_step_unsupported` for three `bounds_out_of_order` fixtures, and the message layer is changed to satisfy them. The retired reason code is documented as such rather than removed. A latent bug that only negative steps could reach: `_reindex_array` built `slice(pos, pos + size*step, step)`, and a downward walk reaching the front of the array computes a negative stop, which NumPy reads as counting from the end — `slice(6, -1, -1)` selects nothing where `slice(6, None, -1)` selects seven elements reversed. Both reindex helpers now go through `_positional_slice`. The stride<0 branches of `_intersect_dimension_map` and `iter_chunk_transforms` were written defensively and had never been reachable. They are now, and they were right: the parts-coverage test gains two reversing views, and the seeded sweep generates downward slices (4,352 of 7,200 chains carry one) across every partitioning. At the wrapper boundary the dialect stays NumPy's, which differs in one place: a reversed *positional* interval like `lazy[2:5:-1]` is empty, not an error, because that is what `x[2:5:-1]` means. Only literal coordinates call it a direction error. Tests: the study's recorded TensorStore corpus lands in `test_tensorstore_parity.py` — fifteen desugarings with their domains, offsets and strides, the three rows that discriminate trunc from floor and ceil, both error families, empty-outside-the-domain, five recorded compositions, and the recorded index-array reversal (a negative step over a gathered axis reverses the array rather than attaching a stride). Assisted-by: ClaudeCode:claude-fable-5
d-v-b added a commit
that referenced
this pull request
Jul 31, 2026
`arr[::-1]` reverses. One desugaring rule covers both signs, as TensorStore 0.1.84 does and as ndsel PR #2 now specifies: omitted bounds resolve on the side the traversal starts and stops (`hi-1` and `lo-1` going down), the source interval is [start, stop) going up and [stop+1, start+1) going down, an empty interval is legal at any coordinate, an interval running the wrong way is an error rather than a silent empty, and the origin is trunc(start/step) for either sign. The corpus is re-vendored from ndsel 92d6a32 in this same commit, because it is the definition of correct here: `slice.json` gains ten negative-step fixtures, `errors.json` retires `negative_step_unsupported` for three `bounds_out_of_order` fixtures, and the message layer is changed to satisfy them. The retired reason code is documented as such rather than removed. A latent bug that only negative steps could reach: `_reindex_array` built `slice(pos, pos + size*step, step)`, and a downward walk reaching the front of the array computes a negative stop, which NumPy reads as counting from the end — `slice(6, -1, -1)` selects nothing where `slice(6, None, -1)` selects seven elements reversed. Both reindex helpers now go through `_positional_slice`. The stride<0 branches of `_intersect_dimension_map` and `iter_chunk_transforms` were written defensively and had never been reachable. They are now, and they were right: the parts-coverage test gains two reversing views, and the seeded sweep generates downward slices (4,352 of 7,200 chains carry one) across every partitioning. At the wrapper boundary the dialect stays NumPy's, which differs in one place: a reversed *positional* interval like `lazy[2:5:-1]` is empty, not an error, because that is what `x[2:5:-1]` means. Only literal coordinates call it a direction error. Tests: the study's recorded TensorStore corpus lands in `test_tensorstore_parity.py` — fifteen desugarings with their domains, offsets and strides, the three rows that discriminate trunc from floor and ceil, both error families, empty-outside-the-domain, five recorded compositions, and the recorded index-array reversal (a negative step over a gathered axis reverses the array rather than attaching a stride). Assisted-by: ClaudeCode:claude-fable-5
d-v-b added a commit
that referenced
this pull request
Aug 12, 2026
… arrays (zarr-developers#4222) * feat(zarr-indexing): LazyArray — generic lazy indexing over array-API arrays A generic wrapper for any array-API-like source (numpy, zarr, cupy, ...) adding TensorStore-style lazy indexing with a positional NumPy dialect: eager __getitem__, .lazy/.oindex/.vindex composing transforms without data access, result()/__array__ materializing. Resolution is partition-based: parts() iterates the base array's partitions projected through the view as resolvable sub-LazyArrays (Partition carries global box coordinates, out placement, and completeness); with_parts() re-partitions the same base explicitly; a single lowering engine serves both partitioned and whole-array sources. Partitioning is discovered from the source (read_chunk_sizes, .chunks) or declared, and never surfaced as a chunks vocabulary. Box selections (no index arrays; affine, interval-representable) are first-class: is_box, bounding_box() (exact hull up to stride), and strides() complete the slab-read story; the design-notes page records the box-vs-query taxonomy and the relationship to TensorStore. Dunders: __dask_tokenize__ (deterministic, canonical-ndsel-body-based), __len__, __iter__, 0-d conversions, pickling. Degenerate all-singleton index-array maps now collapse to constant maps in the transform algebra, and NumPy advanced-index placement rules are implemented faithfully. Assisted-by: ClaudeCode:claude-fable-5 * feat(zarr-indexing): negative-step slices, per merged ndsel 1.0-draft.2 `arr[::-1]` reverses. One desugaring rule covers both signs, as TensorStore 0.1.84 does and as ndsel PR #2 now specifies: omitted bounds resolve on the side the traversal starts and stops (`hi-1` and `lo-1` going down), the source interval is [start, stop) going up and [stop+1, start+1) going down, an empty interval is legal at any coordinate, an interval running the wrong way is an error rather than a silent empty, and the origin is trunc(start/step) for either sign. The corpus is re-vendored from ndsel 92d6a32 in this same commit, because it is the definition of correct here: `slice.json` gains ten negative-step fixtures, `errors.json` retires `negative_step_unsupported` for three `bounds_out_of_order` fixtures, and the message layer is changed to satisfy them. The retired reason code is documented as such rather than removed. A latent bug that only negative steps could reach: `_reindex_array` built `slice(pos, pos + size*step, step)`, and a downward walk reaching the front of the array computes a negative stop, which NumPy reads as counting from the end — `slice(6, -1, -1)` selects nothing where `slice(6, None, -1)` selects seven elements reversed. Both reindex helpers now go through `_positional_slice`. The stride<0 branches of `_intersect_dimension_map` and `iter_chunk_transforms` were written defensively and had never been reachable. They are now, and they were right: the parts-coverage test gains two reversing views, and the seeded sweep generates downward slices (4,352 of 7,200 chains carry one) across every partitioning. At the wrapper boundary the dialect stays NumPy's, which differs in one place: a reversed *positional* interval like `lazy[2:5:-1]` is empty, not an error, because that is what `x[2:5:-1]` means. Only literal coordinates call it a direction error. Tests: the study's recorded TensorStore corpus lands in `test_tensorstore_parity.py` — fifteen desugarings with their domains, offsets and strides, the three rows that discriminate trunc from floor and ceil, both error families, empty-outside-the-domain, five recorded compositions, and the recorded index-array reversal (a negative step over a gathered axis reverses the array rather than attaching a stride). Assisted-by: ClaudeCode:claude-fable-5 * polish(zarr-indexing): re-review minors — step-zero ValueError, kw-only Partition, strides docs - slice step zero now raises ValueError, matching NumPy in the wrapper's positional dialect (was IndexError) - Partition is keyword-only: box was inserted mid-field-list, so positional construction would silently misbind - strides() documents the empty-box case (bounding_box None, strides still defined) Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): plain technical language throughout Rewrite the package's documentation surfaces — docs/index.md, docs/design-notes.md, docs/ndsel.md, docs/api/index.md, the LazyArray, boundary, and transform docstrings, and the 267 changelog fragment — in plain declarative English. Metaphor, personification, rhetorical framing, and emphasis used for effect are replaced with statements of the same technical content. No technical claim, API name, example, or example output changes. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): American spelling (flavour -> flavor) Assisted-by: ClaudeCode:claude-fable-5 * ci(zarr-indexing): scoped lint ignores for the deliberate blind excepts A new ruff release (the CI job floats via uvx) flags BLE001/S110 at the chunk-discovery tolerance and tokenize-fallback sites. Both catches are intentional contracts: discovery must degrade to no-information on any foreign-object failure, and a token call must never raise. Configured as per-file-ignores rather than noqa comments because the pinned pre-commit ruff strips the comments as unused (RUF100) while the floating CI ruff requires them. Assisted-by: ClaudeCode:claude-fable-5 * ci(zarr-indexing): pin ruff in the lint job and justfile Mirrors the main-branch pin (d-v-b#271) so this PR's workflow runs the same ruff version; bump together with the pyproject pin. Assisted-by: ClaudeCode:claude-fable-5 * docs: add lazy-indexing examples for NumPy and Dask Two runnable examples in the house style: wrapping a NumPy array in LazyArray (attribute forwarding, composing selections, box vs query selections, partitions), and using a LazyArray with Dask (from_array, one task per partition, deterministic tokens). Assisted-by: ClaudeCode:claude-fable-5 * docs: compare dask task graphs with fused transforms in the dask example Adds a timed comparison of chained selections through dask.array against the same selections composed into one transform, and a section on when each is the right tool: dask's graph earns its cost when there is computation across chunks, and is overhead when it only defers indexing. Assisted-by: ClaudeCode:claude-fable-5 * test: run examples against this repository's local packages The example runner rewrote only the `zarr` dependency to the local checkout, so an example depending on an in-repo package resolved it from git main instead — and the lazy-indexing examples failed in CI, since LazyArray is not on main yet. Rewrite every package this repository ships, leaving dependencies an example does not declare alone. Assisted-by: ClaudeCode:claude-fable-5 * feat(zarr-indexing): negotiate what indexing a source supports `LazyArray` assumed every wrapped array could do basic slicing and did all fancy work itself, reading a block and post-indexing it with NumPy. That over-reads when the source could gather natively, and it walks a source axis by axis with `take` where one request would do. Each wrapper now carries an `IndexingSupport` level — BASIC, OUTER, OUTER_1VECTOR, VECTORIZED, the taxonomy and member names of xarray's `IndexingSupport` — and every read is split into the largest part of the selection that level can express, asked of the source in one call through `oindex`/`vindex` when it has them, and a residual transform applied to the block that comes back. The split is applied per partition as well as per whole-array read, so a part costs one request. The level is resolved at construction: an explicit `with_indexing_support` wins, then the source's own `__zarr_indexing_support__` (read defensively), then conservative inference — a NumPy array or zarr's `oindex`/`vindex` pair reads as VECTORIZED, everything else as BASIC, the only assumption that is always correct. A multi-array outer request only ever goes to an `oindex` accessor, because a bare `__getitem__` key with two arrays means an outer product to HDF5 and a correlated gather to NumPy. The level decides how much data crosses the boundary, never what `result()` returns. Tests hold that invariant directly: every selection case, at all four levels, against NumPy and zarr sources, partitioned and not; plus test doubles that raise when handed a key their declared level forbids, so exceeding a declaration fails loudly rather than working by accident. Assisted-by: ClaudeCode:claude-fable-5 * fix(zarr-indexing): token the data, not how it is read The token included the partitioning while deliberately excluding the indexing-support level, though both are read strategies that leave the values unchanged. Excluding both means two wrappers that describe the same data token alike, so a consumer caching on tokens reuses one result across partitionings and support levels. Assisted-by: ClaudeCode:claude-fable-5 * fix(zarr-indexing): an empty downward walk selects nothing A slice with a negative step whose start lies before the front of the axis selects nothing, but the positional slice was written as `slice(start, stop, step)` with a negative stop, which NumPy reads as counting from the end: an empty selection of a fancy axis returned the whole axis reversed, and the correlated form raised a broadcast error. Write an empty selection out explicitly. The randomized basic-selection generator drew negative-step starts from `[0, size)` only, so a start before the front of the axis was unreachable and the suite could not see this. It now draws from below `-size` as well, and three cases pin the behavior directly. Assisted-by: ClaudeCode:claude-fable-5 * fix(zarr-indexing): a selection of slices is not a fancy selection An `oindex`/`vindex` step whose entries are all slices carries no coordinates: it narrows the view's own axes and must compose like basic indexing. `_reindex_array_oindex` instead applied each entry positionally to the corresponding axis of the existing index array, without asking whether that axis is one the array varies over or a singleton it merely broadcasts along — the distinction its basic-indexing sibling `_reindex_array` has always made. A slice starting past 0 therefore indexed a size-1 broadcast axis out of range and truncated the whole index array to size 0. A view with no coordinates left resolves to no parts, and `result()` handed back its unwritten `np.empty` buffer: live, on the default path, for any source that advertises chunks. `_reindex_array_oindex` now takes the `ArrayMap` and applies an entry only along its dependency axes (plus the `input_dimension` that breaks the tie for a degenerate length-1 orthogonal selection), preserving a broadcast singleton whatever the slice says. Coordinates never reach a broadcast axis — `_guard_fancy_after_fancy` still rejects genuine fancy-after-fancy with `NotImplementedError`, now under test. Four defects from the same review ride along: - `_array_map_dependency_axes` counted a length-**0** axis as an axis the array varies over, so an empty orthogonal selection classified as correlated and `array_map_dependent_axis` rejected it — a raise on the unpartitioned path where every partitioned path returned the right empty answer. An axis of size 0 carries no dependency any more than a singleton does. - `parts()` raised on a view emptied by a slice over an axis of extent 1. A correlated selection of one point normalizes to an all-singleton index array, so emptying the domain leaves the array at size 1 and the resolver went looking for a chunk. An empty input domain now yields no parts and meets no output domain, matching `result()`. - `sub_transform_to_selections` built `slice(stop + 1, start + 1, stride)` for a negative stride — endpoints swapped, step still negative, so it selected nothing where the reversed axis was meant. Both branches now lower through `_positional_slice`, the same walk an `ArrayMap` axis is reindexed by, which knows a downward walk reaching the front must stop at `None`. - `compose()` evaluated an inner index array over `range(size)` rather than over the outer domain's own range, and addressed it from 0 rather than from the inner domain's origin. Every coordinate resolved to the wrong cell whenever a domain did not start at 0 — which a step-1 slice and a negative-step slice both produce routinely here. - `transform_from_canonical` now rejects a non-integer `index_array` with an `NdselError` carrying `invalid_json`, instead of silently truncating `[0.9, 1.9]` to cells 0 and 1, coercing booleans, or leaking NumPy's own conversion error for strings. The fuzzer missed the first two because `_random_chain` drew at most one fancy step and had no way to spell a step that goes through a fancy accessor while carrying only slices. It now draws such a step separately, and the seeded sweep gained a `parts()` counterpart: `result()` can absorb a defect that the iteration contract cannot, since an empty view assembles correctly from no parts at all. Both sweeps fail on the pre-fix source, as does the new exhaustive stride/extent sweep over the chunk-selection bridge. Assisted-by: ClaudeCode:claude-fable-5 * fix(zarr-indexing): count a domain axis no output map depends on A `vindex` coordinate array with a singleton broadcast axis contributes an axis it does not vary over. A later basic index that consumes the axis it *does* vary over collapses the map to a `ConstantMap` and leaves the broadcast axis in the domain, referenced by nothing. Three places assumed that could not happen: - `sub_transform_to_selections` built `out_selection` with one entry per output map, so a view with such an axis got an index tuple of lower rank than the buffer. `out[out_selection] = value` then placed the part against the leading axes and broadcast the rest — silently wrong data on a partitioned read, and a `parts()` walk that left cells unwritten. - `_restore_domain_axis_order` put an unreferenced axis back as a singleton whatever the domain said. At extent 0 that fabricated a row for a selection whose own `shape` reported it empty. - `_lower_correlated` built its flat gather index from the domain's broadcast shape but added coordinates straight off the stored index array, which is singleton on the axes it does not vary over. The two disagree exactly when a correlated map is constant along a shared broadcast axis. `out_selection` is now built per domain dimension throughout, an unreferenced axis is restored at its own extent, and a correlated map's coordinates are broadcast to the block before being combined. The randomized chain sweep never generated the shape at fault: `_random_vindex` only produced `(length,)` and `(length, 1)` coordinate arrays, neither of which leaves a singleton axis for a later step to strand. It now draws a broadcast rank and places each array's varying axis within it, which reproduces all three failures on the unfixed code. Assisted-by: ClaudeCode:claude-fable-5 * fix(zarr-indexing): a materialized view never hands back the source Five fixes to the wrapper's edges, none of which changes what a selection means. `result()` and `__array__` no longer alias the wrapped array. An unpartitioned read of a basic selection lowers to plain slicing, so it came back as a *view* of the source; NumPy 2 hands whatever `__array__` returns straight to the caller, so `numpy.array(view, copy=True)` aliased it and a write reached through. Under any partitioning the same read allocates, so this also made the answer depend on how the read was divided. The result is now detached whenever it may share memory with the wrapped array, and the `copy=False` refusal no longer justifies itself with a claim the other branch violated. `result()` verifies that the partition walk covered the output before returning it. The buffer is deliberately uninitialized, so any defect in the walk was reported as plausible-looking numbers rather than as an error. The cells each part addresses are counted from the selectors' own shapes — nothing is read — and a walk that does not add up to the view's size raises. Measured on a 16 MiB read: 51 us of accounting against 6.5 ms of read for 64 parts, within noise end to end, and +1.7% at 512 parts. The `BASIC` floor is a promise about the *source*, not about the blocks it returns. The residual is finished with `take`, `reshape` and `transpose`, which were applied to the block unconverted — so a source meeting exactly the documented floor crashed on `oindex[[4, 0, 0], :, :]`. A block that is neither a NumPy array nor an array-API namespace of its own is now coerced, which leaves a device array where it is. `numpy.matrix` is refused at construction: it never reduces rank, so a view's shape and its result disagree on every rank-reducing selection. A `numpy.ma` source keeps its mask through a partitioned read, which allocates a masked buffer. A declaration holding a *foreign* enum member that names one of these four levels — xarray's `IndexingSupport`, whose members these are borrowed from — is honored rather than discarded, since discarding it fell through to inference and answered with a *more* permissive level than the source asked for. `is_complete` is true for a reversing view, which reads every cell of its box back to front; the stride-1 test it failed was about direction, not coverage. `with_parts` accepts `(0,)` and `(0, 0)` for a zero-length axis, which said the same thing as the `()` and uniform spellings it already took, and the positivity error names the working form. Above the token digest limit and without dask, `__dask_tokenize__` returns a value that matches nothing rather than a shape-and-dtype description that two different 4 MiB arrays shared. A cache keyed on it misses instead of lying. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): correct claims a reviewer found false Every statement below was executed before being rewritten, and the replacement was executed too. - "`view + 1`" / "arithmetic materializes through `__array__`" is false. `LazyArray` defines no arithmetic dunders, so `view + 1` raises `TypeError`. What does work is a NumPy *function* — `numpy.add(view, 1)`, `numpy.sum(view)`, `numpy.stack([view, view])` — and an ndarray on the left of the operator. Corrected in the module docstring, `docs/index.md` and the changelog fragment. - "An empty selection returns `None` from both" is false: an empty *box* reports `strides()` and only `bounding_box()` is `None`. The `strides()` docstring already said so; the design notes now agree with it. - "A box touches a contiguous run of parts" is false for a strided box — `[::4]` over 2-wide parts visits every other part. The true property, and the one a partition-walk optimizer would want, is a regularly-spaced run in increasing order, each part at most once. - "TensorStore permits a lower-rank index array" is backwards. Checked against tensorstore 0.1.84: its JSON parser rejects a rank-1 array over a rank-2 domain and accepts full rank with singletons, which is what we emit. *Our* loader is the permissive one. The passage now says both models want full rank, keeps the real rationale (the singletons are what makes the orthogonal/vectorized distinction derivable), and describes our lower-rank acceptance as the compatibility affordance it is. - "Two limits remain" omitted fancy-after-fancy, which is a live `NotImplementedError` reachable from the documented surface, while `index.md` invited chaining fancy steps "anywhere in the chain". Current scope now lists five limits, including the diagonal-view and mixed correlated/orthogonal ones, and both prose pages point at it. - "A single whole-array part stays in the wrapped array's namespace" is only true with *no* partitioning: `result()` branches on whether a partitioning is in force, not on how many boxes it has, so `with_parts((4, 6))` on a 4x6 array returns a plain ndarray. - The changelog stated the support-detection precedence backwards (declaration wins, not inference); `index.md` had a sentence missing its noun; the module docstring's one-line `bounding_box()` summary dropped the stride caveat the three other locations keep; and the package README, the PyPI long description, never mentioned `LazyArray`. Assisted-by: ClaudeCode:claude-fable-5 * fix(zarr-indexing): hold the full-rank invariant inside the engine The index-array rank was checked only from above, so a lower-rank array could exist inside the engine and be read for dependency axes it did not have. Nothing produced one: the tolerance was there for a test asserting compatibility with a body TensorStore itself rejects (verified against 0.1.84 — a rank-1 array over a rank-3 domain is an error in its JSON parser). Require the full input rank in the type, widen a lower-rank array at the JSON boundary where external input arrives, and give the test the shape TensorStore accepts. Assisted-by: ClaudeCode:claude-fable-5 * fix(zarr-indexing): an index array spans the domain it is read over An index array axis must be the domain's extent or a singleton it broadcasts over. Any other size leaves input coordinates with no entry, which read as a smaller selection rather than as the error it is: the truncated array behind one of this review's silent-corruption bugs was a (3, 0) array over a (3, 2) domain, which this rejects at construction. Two fixtures carried the inconsistency they were meant to exercise — an empty array over a domain with room for two coordinates, and a widening case whose array covered three of four positions — and now describe domains their arrays span. Assisted-by: ClaudeCode:claude-fable-5 * test(zarr-indexing): a state machine for chained indexing, and the rank-0 part it found Adds `zarr_indexing.testing`, behind a `testing` extra: a Hypothesis state machine that composes indexing steps onto a LazyArray and checks each step's shape, `result()`, and `parts()` assembly against NumPy, plus the selection strategies on their own. A project can point it at its own array by overriding one method. The machine asserts the documented assembly literally — a part's values must arrive at the shape its out_selection addresses — which is how it found the defect it also fixes: intersecting a correlated transform with a part's bounds collapsed the surviving broadcast block into one axis even when the block was already rank 0, so a view narrowed to a single point produced parts of rank 1. A rank-0 block now stays rank 0, and `result()` drops the reshape that was absorbing the mismatch. Merged from the branch that produced it, which predates the coverage guard in `result()`; the guard stays and the reshape it compensated with goes. Assisted-by: ClaudeCode:claude-fable-5 * fix(zarr-indexing): the defects an adversarial review found at the boundaries Six reviewers went at the package, each proving findings by execution. The algebra held: ~85k chained selections against NumPy, ~24k part assemblies with poisoned buffers, 3.8k transform round-trips evaluated as coordinate maps, all clean. Everything below was at a boundary. `result()` and `__array__(copy=True)` could hand back a live view of the source. `_detach` asked whether the *source* was an ndarray, but a duck array that merely stores its data in NumPy returns NumPy views, and those went straight to the caller — while three docstrings promised the opposite unconditionally. It now asks whether the *result* owns its buffer, so memory is released only when sharing is disproved rather than when it cannot be established. The wire format could not reload its own output. `tolist()` renders every empty array as `[]` once the leading axis is the zero-length one, so an ordinary empty selection lost the axis it varied over, and the loader put it back on a different one by prepending singletons. Nested lists cannot express the shape either, so the body carries it. `index_domain_from_json` was a second undefended way into the same objects: a bare `int()` that truncated 3.9, coerced "3" and True, and let a non-string label into a tuple[str, ...]. It goes through the message layer now, as a transform body always did. With it: a rank ceiling, i64 checks on desugared bounds so normalization stays idempotent, ordered index_array_bounds, and typed errors where raw ones leaked. `sub_transform_to_selections` transposed its blocks two ways. A ConstantMap was emitted as a bare integer, which NumPy counts among the *advanced* indices whenever an index array is present, moving the broadcast axis to the front when a slice separates them — while `out_selection` is built positionally. And the correlated branch documented a points-major block but assembled the chunk selection in output order, so a residual slice before the coordinates arrived slice-major. Constants are length-one slices now, named in drop_axes, and the correlated scatter is permuted to the block NumPy actually returns. Nothing caught either one because `LazyArray` resolves a part through its own lowering and never reads `chunk_selection`; one of the two was even asserted as correct in a passing test's comment. Three further failures were one stale field: `_apply_vindex` carried `input_dimension` onto a map the vindex had just made correlated. The value outlived the shape that justified it and was believed later by a scatter that filed positions under the wrong axis — which is why one view's answer depended on how it was partitioned. The dependency is read back off the array now, and `__post_init__` checks the field it was wrong about: ArrayMap was the one map whose `input_dimension` nothing validated. Also: `oindex` over a correlated view applied its index tuple positionally, NumPy's vectorized rule, collapsing two arrays into one axis; `compose()` indexed an inner array's broadcast singletons by the raw coordinate and sized its one-dimensional shortcut by the input rank while gating on the output rank; and neither checked that the outer transform's output lands inside the inner domain, where a negative would have wrapped. Assisted-by: ClaudeCode:claude-fable-5 * test(zarr-indexing): generate the selections that were never generated A mutation audit ran 31 mutations and 10 survived. They were not scattered: the invariants check thoroughly what a view *returns* and never what it *claims about itself*, and the strategies could not draw whole classes of selection. The generators now draw them. Orthogonal slices carry a step and may stop early, so a strided or reversed slice reaches `oindex` at all. Coordinate lists and masks can be empty, so a fancy selection that selects nothing exists — the shape that lost its axis on the way through JSON. Vectorized coordinate arrays can be multi-dimensional, so a rank-raising `vindex` is generated. Measured over 4000 draws each, every one of those counts was previously 0. The first run of the widened generators found a live defect: an empty Python list carries no element type and NumPy defaults it to float64, so `oindex[[]]` was refused as a non-integer index array. `json.py` already had that case; the boundary did not. NumPy takes `a[np.ix_([])]`, and so does this now. `Partition.is_complete` gets an invariant. It is what a consumer reads to decide it may take a whole-box read, so a wrongly-`True` one is silent corruption — and three separate mutations to `_covers_whole_part` survived the entire suite, including ones reporting `is_complete` for a part carrying two of its three cells. Asserted one way only: the flag is documented as conservative and only the claim to cover everything has to be earned. Two more state machines. The sorted one-dimensional fancy path needs both ranks to be 1, and the output rank is the *source's*, so the rank-3 default source walled it off entirely — 0 hits from the machine against 105 from the unit tests, in the path where reordering and duplicate coordinates are partitioned. A rank-1 source now reaches it 322 times per run. A source with an extent-1 axis covers the other side of a distinction the code draws from the domain rather than the array. Finally the two remaining mutants, both checked by mutating and confirming the failure: the index-array bound checks are probed one past the boundary rather than comfortably outside it, and `_out_selection_cell_count`'s guard is tested directly, since a partition walk only ever produces the forward in-bounds intervals that never reach it. Assisted-by: ClaudeCode:claude-fable-5 * refactor(zarr-indexing)!: settle the API decisions that get dearer after 1.0 `with_parts` decided what it had been given by inspecting the type of it: a sequence of integers meant uniform boxes, a sequence of sequences meant per-axis sizes, and `None` meant no partitioning at all — which also sent `result()` down an entirely different code path. Three semantics behind one parameter, and no way to ask for one of them and be told when you had spelled it wrong. They are now `with_parts`, `with_parts_per_axis` and `unpartitioned`. A harness that draws from a list of mixed partitionings still needs the dispatch, so it exists once, as `zarr_indexing.testing.repartition`. The sizes those methods take are relative to the array being read, not to the view reading it, and a narrowed view partitions the base extents — which could only be discovered from an error message. `base_shape` says it. `ArrayMap`, `IndexTransform` and `Partition` are `frozen=True`, which reads as a promise that a value can be compared and hashed. Both raised: `==` on the index arrays returned an array and then `ValueError: the truth value of an array is ambiguous`, and `hash()` refused an ndarray outright. So no transform could enter a set or key a cache, and the package had already grown an internal `_is_identity_transform` because of it. An `ArrayMap` was frozen but the array inside it was not, so reaching through a view's transform to `index_array[0] = 9` silently changed what the view returned. It is held through a read-only view now — a view rather than a flag on the caller's array, since constructing a map should not take away the right to write to an array you still own. `IndexDomain.narrow` clamped a slice bound to the domain, in a package whose stated invariant is no clamping and no negative wrapping. `narrow(slice(-3, None))` on `[0, 10)` therefore returned the whole axis — reading as the NumPy spelling of "the last three" and answering with something else — and a stop past the end returned a domain its own parent did not contain. Both raise `BoundsCheckError` now, and a stride raises `ValueError` like every other unimplemented request rather than `IndexError`. `Partition.array` was a view of the array while `LazyArray.array` was the raw source: adjacent types, one name, inverted meanings. The part's is `view`. Smaller: `BoundsCheckError` and `VindexInvalidSelectionError` are exported at the top level, being what exported functions raise; `errors.py` no longer claims `zarr.errors` re-exports them by identity, which is false and would have been believed; `parts()` says it is single-use; and `sub_transform_to_selections` says it is provisional rather than implying the rest of the API's stability. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): correct the claims a reviewer could check, and two dialect gaps `boundary.py` justified applying scalars before the advanced indices by asserting NumPy does the same, citing `a[0, [1, 2], :]`. NumPy groups a scalar *with* the advanced indices for placement, so the two disagree the moment a slice separates them: `a[0, ..., [1, 2]]` is `(2, 3)` where `a[0][..., [1, 2]]` is `(3, 2)`. Scalar-first is this package's documented dialect and stays; the reasoning was wrong, and a wrong reason invites someone to "fix" the correct end later. Two places where the dialect really did diverge, both now matching NumPy. A zero-dimensional integer array is a scalar — `a[np.array(2), :]` drops its axis — but only Python and NumPy integers counted, so a 0-d array was widened into a length-1 index array and kept an axis; that was a third answer, agreeing with neither NumPy nor eager zarr. And a multi-dimensional array in an orthogonal selection is refused where the rule lives, instead of surfacing two layers down as a rank complaint about an `index_array` the caller never wrote. `iter_chunk_transforms` documented two shapes for `out_indices` and returns three: the `dict[int, ndarray]` an orthogonal selection with several index arrays produces was missing, from the function downstream integrators use most. Packaging: the README is the PyPI long description, so the monorepo-only development section moved to CONTRIBUTING.md and the dead relative link to the justfile went with it. The examples pinned `zarr-indexing` to a moving `main` rather than to a release they document. Three of the six docs pins claimed to match the repo root and did not, and the justfile's ruff comment contradicted the package's own pin. Assisted-by: ClaudeCode:claude-fable-5 * fix(zarr-indexing): collapse an empty index array instead of extending the format The previous commit fixed the empty-selection round trip by carrying an `index_array_shape` field, on the reasoning that JSON nested lists cannot express the shape of an array with a leading zero axis — `[]` is the only spelling of every empty shape, and `[[]]` is (1, 0) with nothing for (0, 1). That much is true, but the conclusion was wrong: it invented a field ndsel does not define, so the documents this package wrote stopped being ndsel documents. The reference implementation does not have the problem, because it never emits an empty index array. `t[ts.d[0][[]]]` in TensorStore is `out[0] = 0`, emitted as `{}` — a constant map. An empty index array names no cell, and it can only be empty because an input dimension is, since the full-rank invariant makes every axis either 1 or the domain's extent. Nothing is ever read through it, the emptiness is carried by the domain, and the map is degenerate in exactly the way a size-1 array is — which this format already collapses. So it collapses the same way, and the extension is gone. The loader still recovers the axis from the domain for an empty array arriving from a producer that does emit one, since ndsel does not forbid it; that path just no longer has a first-party caller. Checked against tensorstore 0.1.84: every canonical body from 4000 randomized transforms loads into `ts.IndexTransform`, and every one re-emits itself unchanged. No spec change needed. Assisted-by: ClaudeCode:claude-fable-5 * fix(indexing): address lazy array review findings Assisted-by: Codex:gpt-5 * fix(zarr-indexing): keep an empty masked result masked whichever parts are in force An empty view is now answered without reading the source, and the shortcut reached for the array namespace's own `empty`, which knows nothing about masks. An unpartitioned empty view over a masked source therefore came back a plain array while the same view partitioned came back masked — no cells either way, so no value changed, but the caller's type depended on how the read had been divided, which `result()` promises it never does. A masked source goes through `_output_buffer`, which is the branch that knows. Assisted-by: ClaudeCode:claude-fable-5 * docs(indexing): design chunk projection API Assisted-by: Codex:gpt-5.6 * feat(indexing): add reusable chunk plans Assisted-by: Codex:gpt-5.6 * feat(indexing): project chunks through paired transforms Assisted-by: Codex:gpt-5.6 * refactor(indexing): build lazy parts from projections Assisted-by: Codex:gpt-5.6 * refactor(indexing): expose projection-only chunk planning Assisted-by: Codex:gpt-5.6 * docs(indexing): design visual indexing guide Assisted-by: Codex:gpt-5.6 * docs(indexing): explain coordinate origins Assisted-by: Codex:gpt-5.6 * docs(indexing): motivate negative chunk coordinates Assisted-by: Codex:gpt-5.6 * feat(indexing-docs): add SVG diagram renderer Assisted-by: Codex:gpt-5.6 * fix(indexing-docs): harden diagram rendering Assisted-by: Codex:gpt-5.6 * feat(indexing-docs): add accessible guide diagrams Assisted-by: Codex:gpt-5.6 * fix(indexing-docs): correct guide figure semantics Assisted-by: Codex:gpt-5.6 * fix(indexing-docs): prevent selection label overlap Assisted-by: Codex:gpt-5.6 * fix(indexing-docs): validate arrow label offsets Assisted-by: Codex:gpt-5.6 * test(indexing-docs): add executable guide examples Assisted-by: Codex:gpt-5.6 * docs(indexing): add NumPy-first visual tour Assisted-by: Codex:gpt-5.6 * docs(indexing): explain chunk projections visually Assisted-by: Codex:gpt-5.6 * docs(indexing): add indexing and integration references Assisted-by: Codex:gpt-5.6 * fix(indexing-docs): satisfy strict example typing Assisted-by: Codex:gpt-5.6 * docs(indexing): connect visual guide to reference docs Assisted-by: Codex:gpt-5.6 * fix(indexing-docs): source landing quickstart from example Assisted-by: Codex:gpt-5.6 * ci(indexing): verify executable visual docs Assisted-by: Codex:gpt-5.6 * fix(indexing): address visual guide review Assisted-by: Codex:gpt-5.6 * fix(indexing): improve chunk overlay on phones Assisted-by: Codex:gpt-5.6 * docs(indexing): design system-memory chunk cache example Assisted-by: Codex:gpt-5.6 * docs(indexing): introduce half-open intervals Assisted-by: Codex:gpt-5.6 * test(indexing): narrow diagram label elements Assisted-by: Codex:gpt-5.6 * docs(indexing): demonstrate a system-memory chunk cache Assisted-by: Codex:gpt-5.6 * chore(indexing): stop tracking design specs Assisted-by: Codex:gpt-5.6 * feat(indexing): apply and invert transforms Assisted-by: Codex:gpt-5.6 * fix(indexing): handle scalar and wide transform coordinates Assisted-by: Codex:gpt-5.6 * docs(indexing): explain the chunk-cache lifecycle Assisted-by: Codex:gpt-5.6 * fix(indexing): keep lifecycle diagram readable Assisted-by: Codex:gpt-5.6 * fix(indexing): keep lifecycle caption stationary Assisted-by: Codex:gpt-5.6 * fix(indexing): keep guide diagrams readable Assisted-by: Codex:gpt-5.6 * test(indexing): enforce unique guide figure wrappers Assisted-by: Codex:gpt-5.6 * test(indexing): scan all guide sources for figure duplicates Assisted-by: Codex:gpt-5.6 * docs(indexing): distinguish cache indexing modes Assisted-by: Codex:gpt-5.6 * docs(indexing): scope lazy examples to package Assisted-by: Codex:gpt-5.6 * fix(indexing): expose docs modules to root tests Assisted-by: Codex:gpt-5.6 * docs(indexing): omit text from diagram legends Assisted-by: Codex:gpt-5.6 * docs(indexing): strengthen chunk outlines Assisted-by: Codex:gpt-5.6 * docs(indexing): label unselected chunk cells Assisted-by: Codex:gpt-5.6 * fix(indexing): show coordinates in basic selection Assisted-by: Codex:gpt-5.6 * docs(indexing): clarify coordinate-value mapping Assisted-by: Codex:gpt-5.6 * docs(indexing): simplify half-open intervals Assisted-by: Codex:gpt-5.6 * docs(indexing): explain ordered concatenation Assisted-by: Codex:gpt-5.6 * docs(indexing): consolidate visual guide Assisted-by: Codex:gpt-5.6 * docs(indexing): clarify basic selection figure Assisted-by: Codex:gpt-5.6 * docs(indexing): simplify coordinate introduction Assisted-by: Codex:gpt-5.6 * docs(indexing): explain result axis construction Assisted-by: Codex:gpt-5.6 * docs(indexing): enclose slice result axis Assisted-by: Codex:gpt-5.6 * fix(indexing): clarify result array comparison Assisted-by: Codex:gpt-5.6 * fix(indexing): preserve tutorial result ranks Assisted-by: Codex:gpt-5.6 * docs(indexing): promote chunk cache example Assisted-by: Codex:gpt-5.6 * docs(indexing): render chunk cache source Assisted-by: Codex:gpt-5.6 * fix(indexing): close final correctness gaps Assisted-by: Codex:gpt-5.6 * docs(indexing): replace diagrams with ascii Assisted-by: Codex:gpt-5.6 * chore(indexing): remove svg diagram pipeline Assisted-by: Codex:gpt-5.6 * docs(indexing): simplify guide navigation Assisted-by: Codex:gpt-5.6 * fix(indexing): align selection diagram columns Assisted-by: Codex:gpt-5.6 * feat(indexing): add explicit array readers Assisted-by: Codex:gpt-5.6 * refactor(indexing): resolve lazy arrays through readers Assisted-by: Codex:gpt-5.6 * docs(indexing): exercise readers in chunk cache example Assisted-by: Codex:gpt-5.6 * fix(indexing): preserve cache request event ordering Assisted-by: Codex:gpt-5.6 * refactor(indexing): remove indexing capability taxonomy Assisted-by: Codex:gpt-5.6 * docs(indexing): explain explicit reader execution Assisted-by: Codex:gpt-5.6 * test(indexing): avoid constructor spelling assertion Assisted-by: Codex:gpt-5.6 * fix(indexing): complete reader migration Assisted-by: Codex:gpt-5.6 * fix(indexing): keep reader helpers private Assisted-by: Codex:gpt-5.6 * docs(indexing): finalize reader safety contract Assisted-by: Codex:gpt-5.6 * feat(indexing): add compact chunk grids Assisted-by: Codex:gpt-5.6 * fix(indexing): check affine coordinate arithmetic Assisted-by: Codex:gpt-5.6 * fix(indexing): validate composed constants Assisted-by: Codex:gpt-5.6 * fix(indexing): expose grid size representation Assisted-by: Codex:gpt-5.6 * fix(indexing): validate direct advanced selections Assisted-by: Codex:gpt-5.6 * fix(indexing): handle boolean list masks Assisted-by: Codex:gpt-5.6 * fix(indexing): project sparse affine selections directly Assisted-by: Codex:gpt-5.6 * fix(indexing): normalize chunk planner positions Assisted-by: Codex:gpt-5.6 * fix(indexing): lower reader transforms through bounded slabs Assisted-by: Codex:gpt-5.6 * refactor(indexing): expose global partition read context Assisted-by: Codex:gpt-5.6 * fix(indexing): resolve read context annotations Assisted-by: Codex:gpt-5.6 * feat(indexing): reuse prepared partition plans Assisted-by: Codex:gpt-5.6 * fix(indexing): validate prepared partition coverage Assisted-by: Codex:gpt-5.6 * test(indexing): enforce transform materialization laws Assisted-by: Codex:gpt-5.6 * docs(indexing): clarify chunk projection locality Assisted-by: Codex:gpt-5.6 * fix(indexing): harden transform and partition validation Assisted-by: Codex:gpt-5.6 * docs(indexing): separate examples from snippets Assisted-by: Codex:gpt-5.6 * feat(indexing): support index protocol selectors Assisted-by: Codex:gpt-5.6 * feat(indexing): compose fancy selections without restriction A second oindex/vindex/mask step may now land on any axis of an already-fancy view, including axes an existing index array merely broadcasts along. Array-carrying transforms route through compose() instead of being rewritten in place: the selection is applied to an identity transform over the current domain (same dialect by construction) and chained on, which evaluates the existing index arrays at the new coordinates. The in-place reindex machinery and its fancy-after-fancy guard are deleted. Resolution classifies transforms by structure (the new public index_array_structure): pure per-axis outer products keep the orthogonal resolvers; correlated maps, mixtures, and index arrays sharing an input axis (diagonal gathers) all take the pointwise path, whose intersect and lower stages now broadcast per-map blocks instead of assuming full-block index arrays. Only hand-built affine diagonals (an index array and a slice map bound to the same axis) remain unsupported. This removes the crash where a slice-only vindex step (view.lazy.vindex[...] or vindex[..., scalar]) after a correlated gather misclassified the gather as orthogonal and failed at result(), and fixes a stale input_dimension surviving integer indexing of an empty map's pinned axis. Also, from the same review: result(parts=...) raises ValueError instead of AssertionError when supplied parts do not tile the view; with_parts and with_parts_per_axis raise the documented ValueError for non-iterable input; prepared-part validation uses plain assignment for box parts; and __dask_tokenize__ digests the canonical transform body instead of embedding it, keeping tokens small for large fancy selections. The testing state machine now draws any number of fancy steps per chain. Assisted-by: ClaudeCode:claude-fable-5 * refactor(indexing)!: retire ArrayMap.input_dimension What an index-array map depends on is now read from one place: its full-rank array's shape, whose non-singleton axes are the dependency axes. The retired field pinned the orthogonal axis redundantly and could contradict the array it rode on; every bug found in two adversarial review rounds traced back to its bookkeeping (stale values surviving reindexing, misclassification of correlated maps, dangling axes after integer indexing). The one shape the field disambiguated - a single-coordinate, all-singleton array - is normalized away instead: the selection and composition layers build it as the ConstantMap it equals (output_map.array_map_or_constant), exactly as the JSON serializer has always collapsed it on the wire. A consequence is that a length-1 fancy selection now classifies as a box (is_box, bounding_box, strides), which is the semantically sharper answer. Hand-built all-singleton, empty, or shared-axis ArrayMaps remain valid values and resolve through the pointwise path. Fallout removed with the field: the post-init consistency validation, the basic-indexing renumbering of pinned axes, the JSON loader's global dependency-axis reconstruction, and composition's binding carry-through. The sorted 1-D chunk-planning fast path now applies to either fancy spelling, since the flavors coincide in one dimension. The wire format is unaffected - it never carried the field. BREAKING: ArrayMap.__init__ no longer accepts input_dimension, and the attribute is gone; array_map_dependent_axis now answers from the shape alone. Assisted-by: ClaudeCode:claude-fable-5 * feat(zarr-indexing): a reader for sources that only accept unit-step slices BasicReader reads the minimum by pushing strided and descending selections down as positive-step slices, which assumes the source accepts any step. Integrating a zarrs-backed array showed how common the narrower contract is: FFI bindings and HTTP range endpoints support nothing but slice(start, stop, 1), leaving every such backend to rewrite the same cover-and-restride lowering in its facade. UnitStepReader moves that lowering behind the Reader boundary. The decomposition covers each DimensionMap with the smallest ascending unit-step slab and replays the original stride in the residual — the same move the basic decomposition already makes for direction, extended to magnitude. The residual lowering needed no change: positive strides slice the block, descending ones were always gathered. The cost is explicit in the docstring: a strided selection over-reads its cover by the stride factor, bounded by partitioning the wrapping LazyArray. The existing reader contract cases and the affine-overflow parity test now run across all three built-in readers, through a source that rejects anything but ascending unit-step slices inside its bounds. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): the dense-box re-partition idiom is_box and strides() exist so a consumer can choose a read strategy, but the three lines that act on them were only discoverable by deriving them. The integrations guide now states the policy: a dense box resolves best as one backend slab read — re-partition to the base shape and let the backend dispatch, decode in parallel, and partial-decode shards on its own side — while strided boxes and gathers keep the partitioning, which bounds every cover by one part and makes hull-sized reads of sparse selections structurally impossible. The snippet is executable and pins both regimes by observed reads: the corner gather touches four single cells, never the hull; the dense box is exactly one call. A closing subsection points sources that only accept unit-step slices at unit_step_reader. Assisted-by: ClaudeCode:claude-fable-5 * test(zarr-indexing): re-vendor ndsel conformance corpus at 49b9e1db Re-vendor from zarr-developers/ndsel main (49b9e1db1ca93c55f320b025a666367de87a9014, merge of ndsel PR #3). Only transform.json changed vs the previously vendored 92d6a32d: two new fixtures pin empty index_array serialization — normalize carries an empty index_array verbatim rather than rewriting it to a constant map, while a producer SHOULD collapse it to a constant output map, which zarr_indexing.json already does. All other corpus files are byte-identical. Assisted-by: ClaudeCode:claude-fable-5 * ci(zarr-indexing): run the tensorstore parity tests test_ndsel_tensorstore.py skipped everywhere because tensorstore was in no dependency group or workflow. Add a dedicated step to the test job that overlays tensorstore (>=0.1.84, wheels cover the whole 3.12-3.14 matrix) and runs the two parity modules; the main pytest run stays byte-identical to the local canonical invocation. Verified locally against tensorstore 0.1.85 on CPython 3.14: 88 passed, 0 skipped. Assisted-by: ClaudeCode:claude-fable-5 * test(zarr-indexing): derive the docs include graph instead of registering it The doc-example tests accreted during the documentation build-out as one-off guardrails: hand-maintained registries of snippet regions, exact heading and navigation strings, substring pins on teaching prose, and tombstones for migrations that already happened. Each new snippet had to be registered by hand, and editing a sentence could fail CI. The registries also missed the one failure they existed to prevent: a page including a region nobody registered was invisible to them. The suite now states two kinds of contract and nothing else. Structural: every '--8<--' include in the rendered markdown — discovered by scanning, so new snippets are covered the day they are written — resolves to exactly one file with a balanced, non-empty region, and every snippet executes, its inline assertions serving as the value check where expected values were previously duplicated into test tables. Behavioral: the pattern matrix, the wrapped-source contract, and the chunk-cache lifecycle keep their tests, because an example cannot assert its own error paths. pymdownx.snippets gains check_paths: true, so the strict docs build now fails on an unresolvable include instead of silently rendering nothing. Verified both directions: a deliberately broken region name fails the scan test (naming page, file, and region) and the docs build. Editorial choices — section order, wording, nav — return to review, where they belong. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): lead every page with the simple idea The guide opened with two meta-paragraphs and an annotated table of contents; the one-sentence mental model — lazy indexing builds a view, planning partitions it, result() materializes it — sat below them. It opens with that sentence now, and says plainly that the first four sections serve anyone indexing arrays while the last two serve integrators, so most readers know they can stop early. Advanced material moves out of the beginner path. Negative-origin domains, grid prepending, and the EdgeDimensionGrid/DimensionGridLike comparison sat in section two, before the reader had met a transform; they now live in the design notes as 'Negative-origin domains and prependable grids', linked from the two places that want them. The paired-projection section introduces the cell domain concretely — a table with one row per selected cell, chunk-local address on one side, result position on the other — before naming it. The landing page gains the missing why: many arrays support only plain slicing, and this package grafts the full NumPy dialect onto them. The pattern reference leads with the selection matrix readers come for and moves the positions-vs-literal-coordinates table after it. The materialization warning becomes a list, and captions that narrated their own code are cut. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): document every public method of the public API Every public class, method, and property reachable from the package root now carries a docstring stating its contract: the coordinate frame it speaks (global source, chunk-grid, or zero-origin chunk-local), whether a chunk length is the declared codec size or the boundary-clipped data extent, what is bounds-checked and what extrapolates, and which inputs raise which errors. Protocol members (DimensionGridLike, DimensionGrid) are written as implementer obligations, since the docstring is the contract a third-party grid must satisfy. Dataclass-generated __init__ methods are left to their class docstrings; adding a docstring there would mean hand-writing the constructor for no behavioral reason. Docstrings only — no code, signature, or existing-docstring changes. Verified: an introspection audit over __all__ reports zero public members without docstrings; the full suite and pyright are unchanged. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): attribute docstrings for every public field The previous pass documented methods and properties; dataclass and TypedDict fields — IndexDomain.exclusive_max and 41 siblings across fifteen classes — carried no per-attribute documentation, only prose in their class docstrings. Each public field now has an attribute docstring stating what the value means, its coordinate frame or units, and the invariant it carries (literal bounds may be negative, edges are declared codec sizes unclipped by extent, derived fields say what they are derived from, wire bounds admit the infinities the engine refuses to lower). Found by an AST audit, since attribute docstrings are invisible to runtime introspection; that audit now reports zero undocumented public fields. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): the input/output-to-request/source dictionary, and each map kind in NumPy terms An IndexTransform is a function between coordinate spaces, and its fields speak function vocabulary (input, output) while every array-minded reader speaks request and source. The confusion this causes is concentrated in one word: 'output' looks like data, but names the output side of the coordinate function — which is where values are read FROM, since data flows against the arrow. The transform section of the guide and the transform API page now state the dictionary outright, in a two-row table, at the moment a reader first meets the fields, along with why the neutral names exist: composition, where an interior transform has neither a request nor a source side. The three output map kinds are now demonstrated executably against their NumPy counterparts: DimensionMap against basic and negative-step slices, ArrayMap against fancy indexing with order and duplicates preserved, and ConstantMap against numpy.broadcast_to — stated as the value-faithful counterpart precisely because no NumPy selection spells a retained constant axis; an integer index drops it, and a repeated fancy index matches the values while degrading the description to a coordinate list. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): flat landing page — motivation, example, links The grid cards misrendered (misaligned card bodies) and earned their keep poorly: two navigation targets dressed as a layout feature. The landing page now follows the shape convention of projects like pydantic — motivation paragraph, install, one quickstart with a sentence stating the lazy/eager boundary, then a single annotated link list. The two cards' start-here targets survive as the visual guide entry, which names both audiences and their entry points in one line each. Assisted-by: ClaudeCode:claude-fable-5 * fix(zarr-indexing): a single-point bounds error never mentions a batch apply() delegates to the vectorized kernel shared with apply_many() — the right direction, since the batch path is the hot one — but the kernel's diagnostic leaked through it: apply((11,)) on a [-10, 10) domain reported 'point at batch position ()', naming a batch the caller never formed, from a private frame the caller never called. The kernel now raises an internal structured signal (dimension, value, bounds, batch position) and each public entry formats it in its own vocabulary, 'from None' so the traceback ends at the API layer: apply says 'coordinate 11 on input dimension 0 is outside the domain [-10, 10)'; apply_many keeps the batch-position form, where that context is exactly right. Message-only change; BoundsCheckError remains the type on both paths. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): annotation syntax in docstring type slots Parameters and Returns entries now state types as annotations — Sequence[int], tuple[int, ...], numpy.typing.NDArray[numpy.intp] — instead of prose like 'sequence of int' or bare 'tuple'. The annotation is the type's one precise spelling, matches the signature beside it, and names the exact shape where prose left it to the description (both boundary functions' bare 'tuple' entries now state their element structure). Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): the chunk cache example is named for what it is The example was born napari_chunk_cache, and the docs spent two bold disclaimers insisting it is a napari-like consumer, not a napari integration — while the nav entry, section headings, and class names had already settled on 'system-memory chunk cache'. A name that needs disclaimers is the wrong name. The directory, script, and docs page are now system_memory_chunk_cache, matching everything else; napari remains where it belongs, in prose, as the motivating access pattern. Path-only rename: no code, region names, or prose claims change. The derived include-graph test and the strict docs build (check_paths) verify every include and link followed the move. Assisted-by: ClaudeCode:claude-fable-5 * fix(zarr-indexing): an empty domain reads as empty through every reader Composing a fancy selection onto an empty-domain view emits an ArrayMap that is legitimately empty along the vanished axis — a shape the package promises resolves like any other. The resolvers broke that promise: _correlated_map_coords tried to reshape the 0-size array to its non-zero singleton block axes and raised ValueError from all three built-in readers on a direct read_into, a sequence the pre-composition engine handled. LazyArray.result() masked it only through its own size-0 short-circuit. _lower now answers an empty domain first — nothing is selected, so no resolver needs to evaluate maps that may be empty along vanished axes — and the correlated path independently returns no coordinates for an empty broadcast block. Found by an adversarial review fuzzing composed selection chains (3 of 400 random chains hit it); the regression test pins the exact public-API reproduction across all three readers. Assisted-by: ClaudeCode:claude-fable-5 * test(zarr-indexing): execute the CLI examples; strict builds guard anchors and nav Two claims from the doc-test restructure were false, and an adversarial review proved both empirically. First, the lazy_indexing_* examples were said to run under the repository-root example runner; that runner globs only the root examples directory, so the two CLI examples ran under no test at all. They now run as subprocesses here, the dask one skipping where dask is absent. Second, the module docstring claimed mkdocs --strict covered the deleted anchor and nav guards; a strict build passed with a deliberately broken cross-page anchor and with a page omitted from nav, because both are INFO-level by default. mkdocs.yml now sets those validations to warn, which strict promotes to errors — verified failing on a broken anchor and passing clean. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): the vindex error's real contract, and neutral map vocabulary VindexInvalidSelectionError's docstring claimed it covered every non-coordinate vindex form; in fact only the wrapper's validation raises it, only for slices — other invalid entries raise plain IndexError, and the engine-level IndexTransform.vindex accepts residual slice dimensions without raising. The docstring now states the actual raise site. The transform-algebra docstrings (transform, output_map, composition, json) also drop 'storage' for neutral input/output vocabulary: a transform's output side is just output coordinates — in a composition chain an interior transform has no storage side at all. The request/source/storage translation stays where it belongs, in the guide's vocabulary table and the endpoint layers (readers, grids) that really do face arrays. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): an executable Examples section on every public object Every public class and function in the package root now carries a numpydoc Examples section in doctest form: 37 new examples, each a small intuition-builder in the shape the IndexTransform walk-through set — the domain is the result's coordinates, the output maps are the rule, and where an object corresponds to a NumPy indexing concept the example demonstrates the equivalence (DimensionMap against a slice, ArrayMap against fancy indexing with duplicates surviving, ConstantMap against numpy.broadcast_to, compose against chained slicing). The error classes demonstrate their actual raise; the wire types round-trip real bodies. The examples are enforced, not decorative — and closing that loop exposed that the package's existing doctests were never collected anywhere: the package pyproject shadows the repository root's pytest configuration, and no invocation named src. The package config now enables --doctest-modules with the root's option flags, testpaths includes src/zarr_indexing, and the justfile recipe and CI workflow collect it explicitly. 1280 tests pass, 45 of them doctests. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): apply and apply_many say what they locate, not their signature 'Map one input coordinate to an output coordinate' restates the type signature in prose — any function maps inputs to outputs. The summaries now speak the class docstring's array-indexing frame: apply maps a coordinate of the domain (a result cell) to the source coordinate its value is read from, by evaluating each output map; apply_many is the batch form. Both gain a doctest locating cells of the [::2] transform, and both state that no data is touched — this is the coordinate arrow, running result to source. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): method summaries never lean on the ambiguous naked input/output 'input' and 'output' are reserved algebra terms in this package, and in a method summary they collide with ordinary function-speak: identity's 'input coordinate i maps to output coordinate i' reads equally as the algebra statement and as a vacuous description of any function — the same trap apply's summary fell into. identity, intersect, and translate now speak the array frame instead: every result cell reads the source at its own address; keep only the cells whose source coordinates fall inside the box; shift the source coordinates every cell reads. Field docstrings and class summaries keep the naked terms where no call is in sight and the technical reading is the only one available. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): drop non-behavior statements that disambiguate nothing 'No data is touched' in apply/apply_many and 'without I/O' on the oindex/vindex accessors stated what the functions do not do without clarifying what they do: nothing about a coordinate lookup, or about an accessor documented to return a new transform, suggests data movement. Removed. The statements that earn their negation stay: the module thesis, the class contract, and __getitem__ — where subscription syntax genuinely suggests an eager read to anyone arriving from zarr. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): a flat sidebar — sections must earn their existence 'Use lazy indexing' held one page: a disclosure triangle and a competing label with no organization gained. 'Practical reference' classified nothing and grouped two pages serving different audiences — the exact split the guide's opening and the landing page's annotated links already route explicitly. Both dissolve into top-level entries. Examples and API Reference keep their sections, being the only real collections at this site's size; the ndsel wire format and design notes gain explicit labels and sit in the for-builders tail before the API. Assisted-by: ClaudeCode:claude-fable-5 * docs(zarr-indexing): the three guide pages are one Guide section Visual guide, indexing patterns, and integration boundaries sat at top level with the same rank as the fifteen-page API Reference — three pages wearing category clothes. They are one collection, and the docs/guide/ directory said so all along: learn it, look it up, apply it at the…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps fsspec from 2021.6.0 to 2021.6.1.
Commits
540a6f7Merge pull request #676 from skshetry/find-less-isfile-callsa84d7feMerge pull request #680 from jrbourbeau/entrypoints-selectbeade7fMerge pull request #679 from martindurant/mkdir_twice2eb83d0Do not ignote existing dir in local mkdir1e85844add tests384b872Update entrypoint selection0c469bfMerge pull request #678 from d-v-b/sequential_rm700ed8aLocalFileSystem support for a sequence of string arguments66ee7d8avoid isfile calls on find as much as possiblec3dac4fMerge pull request #674 from martindurant/docsDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebasewill rebase this PR@dependabot recreatewill recreate this PR, overwriting any edits that have been made to it@dependabot mergewill merge this PR after your CI passes on it@dependabot squash and mergewill squash and merge this PR after your CI passes on it@dependabot cancel mergewill cancel a previously requested merge and block automerging@dependabot reopenwill reopen this PR if it is closed@dependabot closewill close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually@dependabot ignore this major versionwill close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this minor versionwill close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this dependencywill close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)