Remove ArrayV2 - #1857

Merged
normanrz merged 18 commits into
batched-codec-pipelinefrom
v3-v2-pipeline
May 16, 2024
Merged

Remove ArrayV2#1857
normanrz merged 18 commits into
batched-codec-pipelinefrom
v3-v2-pipeline

Conversation

@normanrz

@normanrznormanrz commented May 8, 2024•
edited
Loading

Copy link
Copy Markdown
Member

This PR removes the ArrayV2 class. The Array class can now take either a ArrayMetadata or ArrayV2Metadata object and perform create, open and save accordingly.
Note, there is still a separate create_v2 method. We might want to merge that into the create method.

The V2 codec pipeline is implemented with the new V2Compressor and V2Filters codecs. They act as wrappers around the compressor and filters of zarr v2.

I also did some light refactorings in the Group to unify with the Array.

cc @jhamman

TODO:

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/tutorial.rst
  • Changes documented in docs/release.rst
  • GitHub Actions have all passed
  • Test coverage is 100% (Codecov passes)

@normanrznormanrz added the V3 label May 8, 2024
@normanrznormanrz changed the title Implement v2 encode/decode with the codec pipelinesRemove ArrayV2May 9, 2024
@normanrz
normanrz requested review from d-v-b and jhammanMay 9, 2024 18:42
@normanrz

normanrz commented May 9, 2024

Copy link
Copy Markdown
MemberAuthor

@jhamman This PR quickly turned into the elimination of the ArrayV2 class. Is there someone already working on porting over the v2 array tests (@d-v-b)?

d-v-b
d-v-b reviewed May 9, 2024
View reviewed changes
Comment threadsrc/zarr/group.py Outdated
d-v-b
d-v-b reviewed May 9, 2024
View reviewed changes
Comment threadsrc/zarr/__init__.py Outdated
d-v-b
d-v-b reviewed May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py Outdated
d-v-b
d-v-b reviewed May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
d-v-b
d-v-b reviewed May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
@normanrz@d-v-b
Apply suggestions from code review
2a64c69
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
@normanrznormanrz mentioned this pull request May 10, 2024
Merged
6 tasks
jhamman
jhamman reviewed May 11, 2024
View reviewed changes

@jhammanjhamman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great @normanrz! I'll give this a deeper look next week but here are some initial reactions.

Comment threadsrc/zarr/array.py Outdated
Comment threadsrc/zarr/array.py Outdated
@normanrz
normanrz requested a review from jhammanMay 13, 2024 19:34
@normanrz
normanrz requested a review from d-v-bMay 13, 2024 20:17
jhamman
jhamman approved these changes May 14, 2024
View reviewed changes

@jhammanjhamman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @normanrz!

Comment threadsrc/zarr/__init__.py Outdated
Comment threadsrc/zarr/metadata.py
@jhammanjhamman mentioned this pull request May 14, 2024
Closed
6 tasks
normanrzand others added 3 commits May 14, 2024 08:24
@normanrz

normanrz commented May 14, 2024

Copy link
Copy Markdown
MemberAuthor

This PR depends on #1670. I'll leave this one open until the other is ready.

@normanrz
normanrz merged commit 95ae4b6 into batched-codec-pipelineMay 16, 2024
@normanrz
normanrz deleted the v3-v2-pipeline branch May 16, 2024 09:51
@jhammanjhamman mentioned this pull request May 17, 2024
Closed
normanrz added a commit that referenced this pull request May 17, 2024
@normanrz@d-v-b@jhamman
Reworked codec pipelines (#1670)
846c085
* merge
* refactors CodecPipelines
* fixes
* adds HybridCodecPipeline
* fixes
* typing
* typing
* consistent naming
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* encode/decode are batched by default
* use zarr.config for batch_size
* don't use global lru_cache
* removes HybridCodecPipeline
* generic codec classes
* default batch size = 1
* default batch size = 1
* docs
* Update src/zarr/codecs/batched_codec_pipeline.py
Co-authored-by: Joe Hamman <joe@earthmover.io>
* mv batched_codec_pipeline -> pipeline
* Remove ArrayV2 (#1857)
* adds wrapper codecs for the v2 codec pipeline
* encode_chunk_key
* refactor ArrayV2 away
* empty zattrs
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* unify ArrayMetadata
* abstract ArrayMetadata
* unified Array.create
* use zarr.config for batch_size
* __init__.py aktualisieren
Co-authored-by: Joe Hamman <joe@earthmover.io>
* ruff
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
* merge
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
@normanrznormanrz mentioned this pull request May 17, 2024
Merged
6 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jhammanjhammanjhamman approved these changes
@d-v-bd-v-bAwaiting requested review from d-v-b

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@normanrz@jhamman@d-v-b
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Remove ArrayV2 - #1857
Merged
normanrz merged 18 commits into
batched-codec-pipelinefrom 
v3-v2-pipeline
May 16, 2024
Merged
Remove ArrayV2#1857
normanrz merged 18 commits into
batched-codec-pipelinefrom 
v3-v2-pipeline
Conversation
@normanrz
@normanrznormanrz
commented
May 8, 2024•

edited

Loading

Copy link


Copy Markdown

Member
This PR removes the ArrayV2 class. The Array class can now take either a ArrayMetadata or ArrayV2Metadata object and perform create, open and save accordingly.

Note, there is still a separate create_v2 method. We might want to merge that into the create method.
The V2 codec pipeline is implemented with the new V2Compressor and V2Filters codecs. They act as wrappers around the compressor and filters of zarr v2.
I also did some light refactorings in the Group to unify with the Array.
cc @jhamman
TODO:
  •  Add unit tests and/or doctests in docstrings
  •  Add docstrings and API docs for any new/modified user-facing classes and functions
  •  New/modified features documented in docs/tutorial.rst
  •  Changes documented in docs/release.rst
  •  GitHub Actions have all passed
  •  Test coverage is 100% (Codecov passes)
@normanrznormanrz
added
the 
V3

label
May 8, 2024
@normanrznormanrz
changed the title
Implement v2 encode/decode with the codec pipelinesRemove ArrayV2May 9, 2024
@normanrz
normanrz
requested review from
d-v-b and jhammanMay 9, 2024 18:42
@normanrz
normanrz
commented
May 9, 2024

Copy link


Copy Markdown

MemberAuthor
@jhamman This PR quickly turned into the elimination of the ArrayV2 class. Is there someone already working on porting over the v2 array tests (@d-v-b)?
d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/group.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/__init__.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
@normanrz@d-v-b
Apply suggestions from code review
2a64c69
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
@normanrznormanrz
mentioned this pull request
May 10, 2024
 Merged

6 tasks
jhamman
jhamman
reviewed
May 11, 2024
View reviewed changes
@jhammanjhamman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great @normanrz! I'll give this a deeper look next week but here are some initial reactions.
Comment threadsrc/zarr/array.py
Outdated

Comment threadsrc/zarr/array.py
Outdated

@normanrz
normanrz
requested a review
from jhammanMay 13, 2024 19:34
@normanrz
normanrz
requested a review
from d-v-bMay 13, 2024 20:17
jhamman
jhamman
approved these changes
May 14, 2024
View reviewed changes
@jhammanjhamman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @normanrz!
Comment threadsrc/zarr/__init__.py
Outdated

Comment threadsrc/zarr/metadata.py
@jhammanjhamman
mentioned this pull request
May 14, 2024
 Closed

6 tasks
normanrzand others
added 3 commits
May 14, 2024 08:24
@normanrz
normanrz
commented
May 14, 2024

Copy link


Copy Markdown

MemberAuthor
This PR depends on #1670. I'll leave this one open until the other is ready.
@normanrz
normanrz
merged commit 95ae4b6
into
batched-codec-pipelineMay 16, 2024
@normanrz
normanrz
deleted the

v3-v2-pipeline

branch
May 16, 2024 09:51
@jhammanjhamman
mentioned this pull request
May 17, 2024
 Closed

normanrz added a commit
that referenced
this pull request
May 17, 2024
@normanrz@d-v-b@jhamman
Reworked codec pipelines (#1670)
846c085
* merge
* refactors CodecPipelines
* fixes
* adds HybridCodecPipeline
* fixes
* typing
* typing
* consistent naming
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* encode/decode are batched by default
* use zarr.config for batch_size
* don't use global lru_cache
* removes HybridCodecPipeline
* generic codec classes
* default batch size = 1
* default batch size = 1
* docs
* Update src/zarr/codecs/batched_codec_pipeline.py
Co-authored-by: Joe Hamman <joe@earthmover.io>
* mv batched_codec_pipeline -> pipeline
* Remove ArrayV2 (#1857)
* adds wrapper codecs for the v2 codec pipeline
* encode_chunk_key
* refactor ArrayV2 away
* empty zattrs
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* unify ArrayMetadata
* abstract ArrayMetadata
* unified Array.create
* use zarr.config for batch_size
* __init__.py aktualisieren
Co-authored-by: Joe Hamman <joe@earthmover.io>
* ruff
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
* merge
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
@normanrznormanrz
mentioned this pull request
May 17, 2024
 Merged

6 tasks
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jhammanjhammanjhamman approved these changes
@d-v-bd-v-bAwaiting requested review from d-v-b

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@normanrz@jhamman@d-v-b

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Remove ArrayV2 - #1857
Merged
normanrz merged 18 commits into
batched-codec-pipelinefrom 
v3-v2-pipeline
May 16, 2024
Merged
Remove ArrayV2#1857
normanrz merged 18 commits into
batched-codec-pipelinefrom 
v3-v2-pipeline
Conversation
@normanrz
@normanrznormanrz
commented
May 8, 2024•

edited

Loading

Copy link


Copy Markdown

Member
This PR removes the ArrayV2 class. The Array class can now take either a ArrayMetadata or ArrayV2Metadata object and perform create, open and save accordingly.

Note, there is still a separate create_v2 method. We might want to merge that into the create method.
The V2 codec pipeline is implemented with the new V2Compressor and V2Filters codecs. They act as wrappers around the compressor and filters of zarr v2.
I also did some light refactorings in the Group to unify with the Array.
cc @jhamman
TODO:
  •  Add unit tests and/or doctests in docstrings
  •  Add docstrings and API docs for any new/modified user-facing classes and functions
  •  New/modified features documented in docs/tutorial.rst
  •  Changes documented in docs/release.rst
  •  GitHub Actions have all passed
  •  Test coverage is 100% (Codecov passes)
@normanrznormanrz
added
the 
V3

label
May 8, 2024
@normanrznormanrz
changed the title
Implement v2 encode/decode with the codec pipelinesRemove ArrayV2May 9, 2024
@normanrz
normanrz
requested review from
d-v-b and jhammanMay 9, 2024 18:42
@normanrz
normanrz
commented
May 9, 2024

Copy link


Copy Markdown

MemberAuthor
@jhamman This PR quickly turned into the elimination of the ArrayV2 class. Is there someone already working on porting over the v2 array tests (@d-v-b)?
d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/group.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/__init__.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
@normanrz@d-v-b
Apply suggestions from code review
2a64c69
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
@normanrznormanrz
mentioned this pull request
May 10, 2024
 Merged

6 tasks
jhamman
jhamman
reviewed
May 11, 2024
View reviewed changes
@jhammanjhamman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great @normanrz! I'll give this a deeper look next week but here are some initial reactions.
Comment threadsrc/zarr/array.py
Outdated

Comment threadsrc/zarr/array.py
Outdated

@normanrz
normanrz
requested a review
from jhammanMay 13, 2024 19:34
@normanrz
normanrz
requested a review
from d-v-bMay 13, 2024 20:17
jhamman
jhamman
approved these changes
May 14, 2024
View reviewed changes
@jhammanjhamman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @normanrz!
Comment threadsrc/zarr/__init__.py
Outdated

Comment threadsrc/zarr/metadata.py
@jhammanjhamman
mentioned this pull request
May 14, 2024
 Closed

6 tasks
normanrzand others
added 3 commits
May 14, 2024 08:24
@normanrz
normanrz
commented
May 14, 2024

Copy link


Copy Markdown

MemberAuthor
This PR depends on #1670. I'll leave this one open until the other is ready.
@normanrz
normanrz
merged commit 95ae4b6
into
batched-codec-pipelineMay 16, 2024
@normanrz
normanrz
deleted the

v3-v2-pipeline

branch
May 16, 2024 09:51
@jhammanjhamman
mentioned this pull request
May 17, 2024
 Closed

normanrz added a commit
that referenced
this pull request
May 17, 2024
@normanrz@d-v-b@jhamman
Reworked codec pipelines (#1670)
846c085
* merge
* refactors CodecPipelines
* fixes
* adds HybridCodecPipeline
* fixes
* typing
* typing
* consistent naming
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* encode/decode are batched by default
* use zarr.config for batch_size
* don't use global lru_cache
* removes HybridCodecPipeline
* generic codec classes
* default batch size = 1
* default batch size = 1
* docs
* Update src/zarr/codecs/batched_codec_pipeline.py
Co-authored-by: Joe Hamman <joe@earthmover.io>
* mv batched_codec_pipeline -> pipeline
* Remove ArrayV2 (#1857)
* adds wrapper codecs for the v2 codec pipeline
* encode_chunk_key
* refactor ArrayV2 away
* empty zattrs
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* unify ArrayMetadata
* abstract ArrayMetadata
* unified Array.create
* use zarr.config for batch_size
* __init__.py aktualisieren
Co-authored-by: Joe Hamman <joe@earthmover.io>
* ruff
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
* merge
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
@normanrznormanrz
mentioned this pull request
May 17, 2024
 Merged

6 tasks
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jhammanjhammanjhamman approved these changes
@d-v-bd-v-bAwaiting requested review from d-v-b

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@normanrz@jhamman@d-v-b

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Remove ArrayV2 - #1857
Merged
normanrz merged 18 commits into
batched-codec-pipelinefrom 
v3-v2-pipeline
May 16, 2024
Merged
Remove ArrayV2#1857
normanrz merged 18 commits into
batched-codec-pipelinefrom 
v3-v2-pipeline
Conversation
@normanrz
@normanrznormanrz
commented
May 8, 2024•

edited

Loading

Copy link


Copy Markdown

Member
This PR removes the ArrayV2 class. The Array class can now take either a ArrayMetadata or ArrayV2Metadata object and perform create, open and save accordingly.

Note, there is still a separate create_v2 method. We might want to merge that into the create method.
The V2 codec pipeline is implemented with the new V2Compressor and V2Filters codecs. They act as wrappers around the compressor and filters of zarr v2.
I also did some light refactorings in the Group to unify with the Array.
cc @jhamman
TODO:
  •  Add unit tests and/or doctests in docstrings
  •  Add docstrings and API docs for any new/modified user-facing classes and functions
  •  New/modified features documented in docs/tutorial.rst
  •  Changes documented in docs/release.rst
  •  GitHub Actions have all passed
  •  Test coverage is 100% (Codecov passes)
@normanrznormanrz
added
the 
V3

label
May 8, 2024
@normanrznormanrz
changed the title
Implement v2 encode/decode with the codec pipelinesRemove ArrayV2May 9, 2024
@normanrz
normanrz
requested review from
d-v-b and jhammanMay 9, 2024 18:42
@normanrz
normanrz
commented
May 9, 2024

Copy link


Copy Markdown

MemberAuthor
@jhamman This PR quickly turned into the elimination of the ArrayV2 class. Is there someone already working on porting over the v2 array tests (@d-v-b)?
d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/group.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/__init__.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
@normanrz@d-v-b
Apply suggestions from code review
2a64c69
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
@normanrznormanrz
mentioned this pull request
May 10, 2024
 Merged

6 tasks
jhamman
jhamman
reviewed
May 11, 2024
View reviewed changes
@jhammanjhamman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great @normanrz! I'll give this a deeper look next week but here are some initial reactions.
Comment threadsrc/zarr/array.py
Outdated

Comment threadsrc/zarr/array.py
Outdated

@normanrz
normanrz
requested a review
from jhammanMay 13, 2024 19:34
@normanrz
normanrz
requested a review
from d-v-bMay 13, 2024 20:17
jhamman
jhamman
approved these changes
May 14, 2024
View reviewed changes
@jhammanjhamman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @normanrz!
Comment threadsrc/zarr/__init__.py
Outdated

Comment threadsrc/zarr/metadata.py
@jhammanjhamman
mentioned this pull request
May 14, 2024
 Closed

6 tasks
normanrzand others
added 3 commits
May 14, 2024 08:24
@normanrz
normanrz
commented
May 14, 2024

Copy link


Copy Markdown

MemberAuthor
This PR depends on #1670. I'll leave this one open until the other is ready.
@normanrz
normanrz
merged commit 95ae4b6
into
batched-codec-pipelineMay 16, 2024
@normanrz
normanrz
deleted the

v3-v2-pipeline

branch
May 16, 2024 09:51
@jhammanjhamman
mentioned this pull request
May 17, 2024
 Closed

normanrz added a commit
that referenced
this pull request
May 17, 2024
@normanrz@d-v-b@jhamman
Reworked codec pipelines (#1670)
846c085
* merge
* refactors CodecPipelines
* fixes
* adds HybridCodecPipeline
* fixes
* typing
* typing
* consistent naming
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* encode/decode are batched by default
* use zarr.config for batch_size
* don't use global lru_cache
* removes HybridCodecPipeline
* generic codec classes
* default batch size = 1
* default batch size = 1
* docs
* Update src/zarr/codecs/batched_codec_pipeline.py
Co-authored-by: Joe Hamman <joe@earthmover.io>
* mv batched_codec_pipeline -> pipeline
* Remove ArrayV2 (#1857)
* adds wrapper codecs for the v2 codec pipeline
* encode_chunk_key
* refactor ArrayV2 away
* empty zattrs
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* unify ArrayMetadata
* abstract ArrayMetadata
* unified Array.create
* use zarr.config for batch_size
* __init__.py aktualisieren
Co-authored-by: Joe Hamman <joe@earthmover.io>
* ruff
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
* merge
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
@normanrznormanrz
mentioned this pull request
May 17, 2024
 Merged

6 tasks
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jhammanjhammanjhamman approved these changes
@d-v-bd-v-bAwaiting requested review from d-v-b

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@normanrz@jhamman@d-v-b

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Remove ArrayV2 - #1857
Merged
normanrz merged 18 commits into
batched-codec-pipelinefrom 
v3-v2-pipeline
May 16, 2024
Merged
Remove ArrayV2#1857
normanrz merged 18 commits into
batched-codec-pipelinefrom 
v3-v2-pipeline
Conversation
@normanrz
@normanrznormanrz
commented
May 8, 2024•

edited

Loading

Copy link


Copy Markdown

Member
This PR removes the ArrayV2 class. The Array class can now take either a ArrayMetadata or ArrayV2Metadata object and perform create, open and save accordingly.

Note, there is still a separate create_v2 method. We might want to merge that into the create method.
The V2 codec pipeline is implemented with the new V2Compressor and V2Filters codecs. They act as wrappers around the compressor and filters of zarr v2.
I also did some light refactorings in the Group to unify with the Array.
cc @jhamman
TODO:
  •  Add unit tests and/or doctests in docstrings
  •  Add docstrings and API docs for any new/modified user-facing classes and functions
  •  New/modified features documented in docs/tutorial.rst
  •  Changes documented in docs/release.rst
  •  GitHub Actions have all passed
  •  Test coverage is 100% (Codecov passes)
@normanrznormanrz
added
the 
V3

label
May 8, 2024
@normanrznormanrz
changed the title
Implement v2 encode/decode with the codec pipelinesRemove ArrayV2May 9, 2024
@normanrz
normanrz
requested review from
d-v-b and jhammanMay 9, 2024 18:42
@normanrz
normanrz
commented
May 9, 2024

Copy link


Copy Markdown

MemberAuthor
@jhamman This PR quickly turned into the elimination of the ArrayV2 class. Is there someone already working on porting over the v2 array tests (@d-v-b)?
d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/group.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/__init__.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
@normanrz@d-v-b
Apply suggestions from code review
2a64c69
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
@normanrznormanrz
mentioned this pull request
May 10, 2024
 Merged

6 tasks
jhamman
jhamman
reviewed
May 11, 2024
View reviewed changes
@jhammanjhamman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great @normanrz! I'll give this a deeper look next week but here are some initial reactions.
Comment threadsrc/zarr/array.py
Outdated

Comment threadsrc/zarr/array.py
Outdated

@normanrz
normanrz
requested a review
from jhammanMay 13, 2024 19:34
@normanrz
normanrz
requested a review
from d-v-bMay 13, 2024 20:17
jhamman
jhamman
approved these changes
May 14, 2024
View reviewed changes
@jhammanjhamman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @normanrz!
Comment threadsrc/zarr/__init__.py
Outdated

Comment threadsrc/zarr/metadata.py
@jhammanjhamman
mentioned this pull request
May 14, 2024
 Closed

6 tasks
normanrzand others
added 3 commits
May 14, 2024 08:24
@normanrz
normanrz
commented
May 14, 2024

Copy link


Copy Markdown

MemberAuthor
This PR depends on #1670. I'll leave this one open until the other is ready.
@normanrz
normanrz
merged commit 95ae4b6
into
batched-codec-pipelineMay 16, 2024
@normanrz
normanrz
deleted the

v3-v2-pipeline

branch
May 16, 2024 09:51
@jhammanjhamman
mentioned this pull request
May 17, 2024
 Closed

normanrz added a commit
that referenced
this pull request
May 17, 2024
@normanrz@d-v-b@jhamman
Reworked codec pipelines (#1670)
846c085
* merge
* refactors CodecPipelines
* fixes
* adds HybridCodecPipeline
* fixes
* typing
* typing
* consistent naming
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* encode/decode are batched by default
* use zarr.config for batch_size
* don't use global lru_cache
* removes HybridCodecPipeline
* generic codec classes
* default batch size = 1
* default batch size = 1
* docs
* Update src/zarr/codecs/batched_codec_pipeline.py
Co-authored-by: Joe Hamman <joe@earthmover.io>
* mv batched_codec_pipeline -> pipeline
* Remove ArrayV2 (#1857)
* adds wrapper codecs for the v2 codec pipeline
* encode_chunk_key
* refactor ArrayV2 away
* empty zattrs
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* unify ArrayMetadata
* abstract ArrayMetadata
* unified Array.create
* use zarr.config for batch_size
* __init__.py aktualisieren
Co-authored-by: Joe Hamman <joe@earthmover.io>
* ruff
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
* merge
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
@normanrznormanrz
mentioned this pull request
May 17, 2024
 Merged

6 tasks
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jhammanjhammanjhamman approved these changes
@d-v-bd-v-bAwaiting requested review from d-v-b

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@normanrz@jhamman@d-v-b

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Remove ArrayV2 - #1857
Merged
normanrz merged 18 commits into
batched-codec-pipelinefrom 
v3-v2-pipeline
May 16, 2024
Merged
Remove ArrayV2#1857
normanrz merged 18 commits into
batched-codec-pipelinefrom 
v3-v2-pipeline
Conversation
@normanrz
@normanrznormanrz
commented
May 8, 2024•

edited

Loading

Copy link


Copy Markdown

Member
This PR removes the ArrayV2 class. The Array class can now take either a ArrayMetadata or ArrayV2Metadata object and perform create, open and save accordingly.

Note, there is still a separate create_v2 method. We might want to merge that into the create method.
The V2 codec pipeline is implemented with the new V2Compressor and V2Filters codecs. They act as wrappers around the compressor and filters of zarr v2.
I also did some light refactorings in the Group to unify with the Array.
cc @jhamman
TODO:
  •  Add unit tests and/or doctests in docstrings
  •  Add docstrings and API docs for any new/modified user-facing classes and functions
  •  New/modified features documented in docs/tutorial.rst
  •  Changes documented in docs/release.rst
  •  GitHub Actions have all passed
  •  Test coverage is 100% (Codecov passes)
@normanrznormanrz
added
the 
V3

label
May 8, 2024
@normanrznormanrz
changed the title
Implement v2 encode/decode with the codec pipelinesRemove ArrayV2May 9, 2024
@normanrz
normanrz
requested review from
d-v-b and jhammanMay 9, 2024 18:42
@normanrz
normanrz
commented
May 9, 2024

Copy link


Copy Markdown

MemberAuthor
@jhamman This PR quickly turned into the elimination of the ArrayV2 class. Is there someone already working on porting over the v2 array tests (@d-v-b)?
d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/group.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/__init__.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
@normanrz@d-v-b
Apply suggestions from code review
2a64c69
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
@normanrznormanrz
mentioned this pull request
May 10, 2024
 Merged

6 tasks
jhamman
jhamman
reviewed
May 11, 2024
View reviewed changes
@jhammanjhamman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great @normanrz! I'll give this a deeper look next week but here are some initial reactions.
Comment threadsrc/zarr/array.py
Outdated

Comment threadsrc/zarr/array.py
Outdated

@normanrz
normanrz
requested a review
from jhammanMay 13, 2024 19:34
@normanrz
normanrz
requested a review
from d-v-bMay 13, 2024 20:17
jhamman
jhamman
approved these changes
May 14, 2024
View reviewed changes
@jhammanjhamman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @normanrz!
Comment threadsrc/zarr/__init__.py
Outdated

Comment threadsrc/zarr/metadata.py
@jhammanjhamman
mentioned this pull request
May 14, 2024
 Closed

6 tasks
normanrzand others
added 3 commits
May 14, 2024 08:24
@normanrz
normanrz
commented
May 14, 2024

Copy link


Copy Markdown

MemberAuthor
This PR depends on #1670. I'll leave this one open until the other is ready.
@normanrz
normanrz
merged commit 95ae4b6
into
batched-codec-pipelineMay 16, 2024
@normanrz
normanrz
deleted the

v3-v2-pipeline

branch
May 16, 2024 09:51
@jhammanjhamman
mentioned this pull request
May 17, 2024
 Closed

normanrz added a commit
that referenced
this pull request
May 17, 2024
@normanrz@d-v-b@jhamman
Reworked codec pipelines (#1670)
846c085
* merge
* refactors CodecPipelines
* fixes
* adds HybridCodecPipeline
* fixes
* typing
* typing
* consistent naming
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* encode/decode are batched by default
* use zarr.config for batch_size
* don't use global lru_cache
* removes HybridCodecPipeline
* generic codec classes
* default batch size = 1
* default batch size = 1
* docs
* Update src/zarr/codecs/batched_codec_pipeline.py
Co-authored-by: Joe Hamman <joe@earthmover.io>
* mv batched_codec_pipeline -> pipeline
* Remove ArrayV2 (#1857)
* adds wrapper codecs for the v2 codec pipeline
* encode_chunk_key
* refactor ArrayV2 away
* empty zattrs
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* unify ArrayMetadata
* abstract ArrayMetadata
* unified Array.create
* use zarr.config for batch_size
* __init__.py aktualisieren
Co-authored-by: Joe Hamman <joe@earthmover.io>
* ruff
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
* merge
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
@normanrznormanrz
mentioned this pull request
May 17, 2024
 Merged

6 tasks
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jhammanjhammanjhamman approved these changes
@d-v-bd-v-bAwaiting requested review from d-v-b

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@normanrz@jhamman@d-v-b

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Remove ArrayV2 - #1857
Merged
normanrz merged 18 commits into
batched-codec-pipelinefrom 
v3-v2-pipeline
May 16, 2024
Merged
Remove ArrayV2#1857
normanrz merged 18 commits into
batched-codec-pipelinefrom 
v3-v2-pipeline
Conversation
@normanrz
@normanrznormanrz
commented
May 8, 2024•

edited

Loading

Copy link


Copy Markdown

Member
This PR removes the ArrayV2 class. The Array class can now take either a ArrayMetadata or ArrayV2Metadata object and perform create, open and save accordingly.

Note, there is still a separate create_v2 method. We might want to merge that into the create method.
The V2 codec pipeline is implemented with the new V2Compressor and V2Filters codecs. They act as wrappers around the compressor and filters of zarr v2.
I also did some light refactorings in the Group to unify with the Array.
cc @jhamman
TODO:
  •  Add unit tests and/or doctests in docstrings
  •  Add docstrings and API docs for any new/modified user-facing classes and functions
  •  New/modified features documented in docs/tutorial.rst
  •  Changes documented in docs/release.rst
  •  GitHub Actions have all passed
  •  Test coverage is 100% (Codecov passes)
@normanrznormanrz
added
the 
V3

label
May 8, 2024
@normanrznormanrz
changed the title
Implement v2 encode/decode with the codec pipelinesRemove ArrayV2May 9, 2024
@normanrz
normanrz
requested review from
d-v-b and jhammanMay 9, 2024 18:42
@normanrz
normanrz
commented
May 9, 2024

Copy link


Copy Markdown

MemberAuthor
@jhamman This PR quickly turned into the elimination of the ArrayV2 class. Is there someone already working on porting over the v2 array tests (@d-v-b)?
d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/group.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/__init__.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
@normanrz@d-v-b
Apply suggestions from code review
2a64c69
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
@normanrznormanrz
mentioned this pull request
May 10, 2024
 Merged

6 tasks
jhamman
jhamman
reviewed
May 11, 2024
View reviewed changes
@jhammanjhamman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great @normanrz! I'll give this a deeper look next week but here are some initial reactions.
Comment threadsrc/zarr/array.py
Outdated

Comment threadsrc/zarr/array.py
Outdated

@normanrz
normanrz
requested a review
from jhammanMay 13, 2024 19:34
@normanrz
normanrz
requested a review
from d-v-bMay 13, 2024 20:17
jhamman
jhamman
approved these changes
May 14, 2024
View reviewed changes
@jhammanjhamman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @normanrz!
Comment threadsrc/zarr/__init__.py
Outdated

Comment threadsrc/zarr/metadata.py
@jhammanjhamman
mentioned this pull request
May 14, 2024
 Closed

6 tasks
normanrzand others
added 3 commits
May 14, 2024 08:24
@normanrz
normanrz
commented
May 14, 2024

Copy link


Copy Markdown

MemberAuthor
This PR depends on #1670. I'll leave this one open until the other is ready.
@normanrz
normanrz
merged commit 95ae4b6
into
batched-codec-pipelineMay 16, 2024
@normanrz
normanrz
deleted the

v3-v2-pipeline

branch
May 16, 2024 09:51
@jhammanjhamman
mentioned this pull request
May 17, 2024
 Closed

normanrz added a commit
that referenced
this pull request
May 17, 2024
@normanrz@d-v-b@jhamman
Reworked codec pipelines (#1670)
846c085
* merge
* refactors CodecPipelines
* fixes
* adds HybridCodecPipeline
* fixes
* typing
* typing
* consistent naming
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* encode/decode are batched by default
* use zarr.config for batch_size
* don't use global lru_cache
* removes HybridCodecPipeline
* generic codec classes
* default batch size = 1
* default batch size = 1
* docs
* Update src/zarr/codecs/batched_codec_pipeline.py
Co-authored-by: Joe Hamman <joe@earthmover.io>
* mv batched_codec_pipeline -> pipeline
* Remove ArrayV2 (#1857)
* adds wrapper codecs for the v2 codec pipeline
* encode_chunk_key
* refactor ArrayV2 away
* empty zattrs
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* unify ArrayMetadata
* abstract ArrayMetadata
* unified Array.create
* use zarr.config for batch_size
* __init__.py aktualisieren
Co-authored-by: Joe Hamman <joe@earthmover.io>
* ruff
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
* merge
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
@normanrznormanrz
mentioned this pull request
May 17, 2024
 Merged

6 tasks
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jhammanjhammanjhamman approved these changes
@d-v-bd-v-bAwaiting requested review from d-v-b

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@normanrz@jhamman@d-v-b

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Remove ArrayV2 - #1857
Merged
normanrz merged 18 commits into
batched-codec-pipelinefrom 
v3-v2-pipeline
May 16, 2024
Merged
Remove ArrayV2#1857
normanrz merged 18 commits into
batched-codec-pipelinefrom 
v3-v2-pipeline
Conversation
@normanrz
@normanrznormanrz
commented
May 8, 2024•

edited

Loading

Copy link


Copy Markdown

Member
This PR removes the ArrayV2 class. The Array class can now take either a ArrayMetadata or ArrayV2Metadata object and perform create, open and save accordingly.

Note, there is still a separate create_v2 method. We might want to merge that into the create method.
The V2 codec pipeline is implemented with the new V2Compressor and V2Filters codecs. They act as wrappers around the compressor and filters of zarr v2.
I also did some light refactorings in the Group to unify with the Array.
cc @jhamman
TODO:
  •  Add unit tests and/or doctests in docstrings
  •  Add docstrings and API docs for any new/modified user-facing classes and functions
  •  New/modified features documented in docs/tutorial.rst
  •  Changes documented in docs/release.rst
  •  GitHub Actions have all passed
  •  Test coverage is 100% (Codecov passes)
@normanrznormanrz
added
the 
V3

label
May 8, 2024
@normanrznormanrz
changed the title
Implement v2 encode/decode with the codec pipelinesRemove ArrayV2May 9, 2024
@normanrz
normanrz
requested review from
d-v-b and jhammanMay 9, 2024 18:42
@normanrz
normanrz
commented
May 9, 2024

Copy link


Copy Markdown

MemberAuthor
@jhamman This PR quickly turned into the elimination of the ArrayV2 class. Is there someone already working on porting over the v2 array tests (@d-v-b)?
d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/group.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/__init__.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
Outdated

d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
d-v-b
d-v-b
reviewed
May 9, 2024
View reviewed changes
Comment threadsrc/zarr/abc/store.py
@normanrz@d-v-b
Apply suggestions from code review
2a64c69
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
@normanrznormanrz
mentioned this pull request
May 10, 2024
 Merged

6 tasks
jhamman
jhamman
reviewed
May 11, 2024
View reviewed changes
@jhammanjhamman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great @normanrz! I'll give this a deeper look next week but here are some initial reactions.
Comment threadsrc/zarr/array.py
Outdated

Comment threadsrc/zarr/array.py
Outdated

@normanrz
normanrz
requested a review
from jhammanMay 13, 2024 19:34
@normanrz
normanrz
requested a review
from d-v-bMay 13, 2024 20:17
jhamman
jhamman
approved these changes
May 14, 2024
View reviewed changes
@jhammanjhamman
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @normanrz!
Comment threadsrc/zarr/__init__.py
Outdated

Comment threadsrc/zarr/metadata.py
@jhammanjhamman
mentioned this pull request
May 14, 2024
 Closed

6 tasks
normanrzand others
added 3 commits
May 14, 2024 08:24
@normanrz
normanrz
commented
May 14, 2024

Copy link


Copy Markdown

MemberAuthor
This PR depends on #1670. I'll leave this one open until the other is ready.
@normanrz
normanrz
merged commit 95ae4b6
into
batched-codec-pipelineMay 16, 2024
@normanrz
normanrz
deleted the

v3-v2-pipeline

branch
May 16, 2024 09:51
@jhammanjhamman
mentioned this pull request
May 17, 2024
 Closed

normanrz added a commit
that referenced
this pull request
May 17, 2024
@normanrz@d-v-b@jhamman
Reworked codec pipelines (#1670)
846c085
* merge
* refactors CodecPipelines
* fixes
* adds HybridCodecPipeline
* fixes
* typing
* typing
* consistent naming
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* encode/decode are batched by default
* use zarr.config for batch_size
* don't use global lru_cache
* removes HybridCodecPipeline
* generic codec classes
* default batch size = 1
* default batch size = 1
* docs
* Update src/zarr/codecs/batched_codec_pipeline.py
Co-authored-by: Joe Hamman <joe@earthmover.io>
* mv batched_codec_pipeline -> pipeline
* Remove ArrayV2 (#1857)
* adds wrapper codecs for the v2 codec pipeline
* encode_chunk_key
* refactor ArrayV2 away
* empty zattrs
* Apply suggestions from code review
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
* unify ArrayMetadata
* abstract ArrayMetadata
* unified Array.create
* use zarr.config for batch_size
* __init__.py aktualisieren
Co-authored-by: Joe Hamman <joe@earthmover.io>
* ruff
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
* merge
---------
Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
Co-authored-by: Joe Hamman <joe@earthmover.io>
@normanrznormanrz
mentioned this pull request
May 17, 2024
 Merged

6 tasks
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jhammanjhammanjhamman approved these changes
@d-v-bd-v-bAwaiting requested review from d-v-b

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@normanrz@jhamman@d-v-b