feat(zarr): warn on stale hand-edited encoding['chunks'] via metadata baseline - #3
Open
d-v-b wants to merge 1 commit into
Open
feat(zarr): warn on stale hand-edited encoding['chunks'] via metadata baseline#3d-v-b wants to merge 1 commit into
d-v-b wants to merge 1 commit into
Conversation
d-v-bforce-pushed
the
claude/zarr-metadata-fragment
branch
from
July 30, 2026 09:08
be29ee6 to
776bd2fCompared-v-bforce-pushed
the
claude/zarr-chunks-drift-warning
branch
from
July 30, 2026 09:10
21939fa to
243f404Compared-v-bforce-pushed
the
claude/zarr-metadata-fragment
branch
from
July 30, 2026 09:21
776bd2f to
7668a40Compared-v-bforce-pushed
the
claude/zarr-chunks-drift-warning
branch
from
July 30, 2026 09:21
243f404 to
03ac970Compared-v-bforce-pushed
the
claude/zarr-metadata-fragment
branch
from
July 30, 2026 09:26
7668a40 to
34e4e81Compared-v-bforce-pushed
the
claude/zarr-chunks-drift-warning
branch
from
July 30, 2026 09:26
03ac970 to
81694c5Compared-v-bforce-pushed
the
claude/zarr-metadata-fragment
branch
from
July 30, 2026 10:03
34e4e81 to
943f9b4Compare… baseline Use the stored zarr_array_metadata document as a provenance baseline on write: if a variable's encoding['chunks'] matches neither the source array's chunk shape nor the variable's current chunking, it was almost certainly hand-edited and is stale, so emit a DeprecationWarning. Plain round-trips, .chunk(...), and overwrite_encoded_chunks=True do not warn. Assisted-by: ClaudeCode:claude-fable-5 Co-authored-by: Claude <noreply@anthropic.com>
d-v-bforce-pushed
the
claude/zarr-chunks-drift-warning
branch
from
July 30, 2026 10:04
81694c5 to
95359efCompare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Summary
Uses the read-only
encoding["zarr_array_metadata"]document (added in the parent PR) as a provenance baseline to warn about stale, hand-editedencoding["chunks"].On write, if a variable still carries the stored metadata document but its
encoding["chunks"]matches neither the source array's chunk shape nor the variable's own current (dask) chunking, it was almost certainly edited by hand and is now stale →DeprecationWarning.What it deliberately does NOT warn on
encoding["chunks"]still equals the document..chunk(...)— changes the dask chunking but leavesencoding["chunks"]untouched (chunks.py:116-117).overwrite_encoded_chunks=True— setsencoding["chunks"]to the variable's current chunking, i.e. consistent with the data being written.Honest scope note
Fully deprecating any use of the
chunkskey is not feasible:overwrite_encoded_chunks=Trueis a legitimate first-party writer ofencoding["chunks"], and at write time it's indistinguishable from a hand-edit by value alone. So this scopes the warning to the genuinely stale/inconsistent case (differs from both the source and the data), which is the actual footgun, rather than "any use of the key."encodingis a plain mutable dict, so this drift-vs-baseline check at the consumption boundary is the most reliable signal available without wrapping the dict.This also generalizes: the same baseline diff could flag user overrides of other derived keys (
compressors,fill_value, …).Testing
test_encoding_chunks_drift_warns: hand-editingencoding["chunks"]on a round-tripped variable warns.test_encoding_chunks_no_drift_no_warn:.chunk(...)then write does not warn.test_manual_chunk(which exercisesoverwrite_encoded_chunks=True) stays green.Stacked on
claude/zarr-metadata-fragment(PR #2).[This is Claude Code on behalf of Davis Vann Bennett]
🤖 Generated with Claude Code