Skip to content

feat: url-pipeline core — parser, adapter ABC, registry, store hooks - #6

Closed
jhamman wants to merge 2 commits into
feature/zipstore-file-likefrom
feature/url-pipeline-core
Closed

feat: url-pipeline core — parser, adapter ABC, registry, store hooks#6
jhamman wants to merge 2 commits into
feature/zipstore-file-likefrom
feature/url-pipeline-core

Conversation

@jhamman

Copy link
Copy Markdown
Owner

Summary

Implements URL pipeline support (https://github.com/jbms/url-pipeline): '|'-chained URLs resolve through pluggable adapters registered under the 'zarr.url_adapters' entry-point group (entry-point name = URL scheme).

  • zarr.abc.url_pipeline: PipelineSegment, AdapterResolution, PipelineContext, URLPipelineAdapter (single-classmethod contract)
  • zarr.storage._url_pipeline: parse_pipeline / resolve_pipeline; the root sub-URL delegates to make_store so existing file/memory/fsspec routing is unchanged
  • registry: register_url_adapter / get_url_adapter / list_url_adapter_schemes (name check only; no adapter imports)
  • make_store/make_store_path route strings containing '|' (or a registered root scheme) through the resolver; residual store paths combine with the user-supplied path
  • StorePath gains a zarr_format attribute (populated by format segments in a follow-up)

For reviewers

This is the second of a series of Pull Requests toward zarr-developers#2943.

Author attestation

  • I am a human, these are my changes, and I have reviewed and understood every change and can explain why each is correct.

TODO:

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/user-guide/*.md
  • Changes documented as a new file in changes/
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

@jhamman
jhammanforce-pushed the feature/url-pipeline-core branch 2 times, most recently from 03bed15 to 5ae451eCompareJuly 27, 2026 02:12
@jhamman
jhammanforce-pushed the feature/zipstore-file-like branch from 7d33da8 to ef00953CompareJuly 27, 2026 15:12
@jhamman
jhammanforce-pushed the feature/url-pipeline-core branch 2 times, most recently from fb71546 to ec2c8bfCompareJuly 27, 2026 19:10
@jhamman
jhammanforce-pushed the feature/zipstore-file-like branch from 54fa1dd to b5f99fbCompareJuly 27, 2026 19:14
@jhamman
jhammanforce-pushed the feature/url-pipeline-core branch from ec2c8bf to f26b436CompareJuly 27, 2026 19:14
…#4187)
Allows constructing a ZipStore from any seekable binary reader, enabling
zip archives on remote storage:
- io objects (BytesIO, fsspec file objects) are used directly
- minimal readers that are not io.IOBase instances and whose read() may
return buffer-protocol objects rather than bytes (e.g.
obstore.ReadableFile) are adapted via a small io.RawIOBase wrapper
when opened for reading
clear()/move() raise NotImplementedError for file-object-backed stores.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@jhamman
jhammanforce-pushed the feature/url-pipeline-core branch from f26b436 to d77e88fCompareJuly 28, 2026 02:33
Implements URL pipeline support (https://github.com/jbms/url-pipeline):
'|'-chained URLs resolve through pluggable adapters registered under the
'zarr.url_adapters' entry-point group (entry-point name = URL scheme).
- zarr.abc.url_pipeline: PipelineSegment, AdapterResolution,
PipelineContext, URLPipelineAdapter (single-classmethod contract)
- zarr.storage._url_pipeline: parse_pipeline / resolve_pipeline; the root
sub-URL delegates to make_store so existing file/memory/fsspec routing
is unchanged
- registry: register_url_adapter / get_url_adapter /
list_url_adapter_schemes (name check only; no adapter imports)
- make_store/make_store_path route strings containing '|' (or a
registered root scheme) through the resolver; residual store paths
combine with the user-supplied path
- StorePath gains a zarr_format attribute (populated by format segments
in a follow-up)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@jhamman
jhammanforce-pushed the feature/url-pipeline-core branch from d77e88f to 10cb73bCompareJuly 28, 2026 12:30
@jhammanjhamman closed this Jul 28, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@jhamman