Thinning down zarr-python #1274

Closed
Closed
Thinning down zarr-python#1274
Labels
help wantedIssue could use help from someone with familiarity on the topic
Milestone
3.0.0

Description

@sanketverma1704
sanketverma1704
opened on Nov 22, 2022

This came up during one of the conversations with @joshmoore. The idea is to extract functionalities not in V2 Spec and their respective tests from zarr-python. E.g. Zarr Specification V2 has the following stores:

  • Directory Store
  • Nested Directory Store
  • Zip file Store
  • Object Store
  • HTTP Store

But in zarr-python, we have additional stores like SQLiteStore, MongoDBStore etc. If we could remove these additional stores from the zarr-python codebase and host them in a separate repo under zarr-developers, it'll help us:

  • Making the code run faster and more efficiently
  • Run fewer tests and faster completion of them
  • Reduce the noise from dependabot
  • Add maintainers for specific stores
  • Focus on a more concise codebase, thus helping us to add additional features when migrating from V2 to V3

If a user/developer wishes to use the removed functionality, they can shoot pip install zarr-sqlitestore or similar.

FYI: A similar repo is in the works for Numcodecs, which is taken care of by @joshmoore and @jakirkham.

Please let me know your thoughts on this. Thanks!

CC: @zarr-developers/python-core-devs

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedIssue could use help from someone with familiarity on the topic

    Type

    No type

    Projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    , 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
     blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

    Skip to content
    Thinning down zarr-python #1274
    Closed
    Closed
    Thinning down zarr-python#1274
    Labels
    help wantedIssue could use help from someone with familiarity on the topic
    Milestone
    3.0.0
    Description
    @sanketverma1704
    sanketverma1704
    opened on Nov 22, 2022
    This came up during one of the conversations with @joshmoore. The idea is to extract functionalities not in V2 Spec and their respective tests from zarr-python. E.g. Zarr Specification V2 has the following stores:
    • Directory Store
    • Nested Directory Store
    • Zip file Store
    • Object Store
    • HTTP Store
    But in zarr-python, we have additional stores like SQLiteStore, MongoDBStore etc. If we could remove these additional stores from the zarr-python codebase and host them in a separate repo under zarr-developers, it'll help us:
    • Making the code run faster and more efficiently
    • Run fewer tests and faster completion of them
    • Reduce the noise from dependabot
    • Add maintainers for specific stores
    • Focus on a more concise codebase, thus helping us to add additional features when migrating from V2 to V3
    If a user/developer wishes to use the removed functionality, they can shoot pip install zarr-sqlitestore or similar.
    FYI: A similar repo is in the works for Numcodecs, which is taken care of by @joshmoore and @jakirkham.
    Please let me know your thoughts on this. Thanks!
    CC: @zarr-developers/python-core-devs
    Metadata
    Metadata
    Assignees
    No one assigned
      Labels
      help wantedIssue could use help from someone with familiarity on the topic
      Type
      No type
      Projects
      Milestone
      Relationships
      None yet
      Development
      No branches or pull requests
      Issue actions
      
, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

      Skip to content
      Thinning down zarr-python #1274
      Closed
      Closed
      Thinning down zarr-python#1274
      Labels
      help wantedIssue could use help from someone with familiarity on the topic
      Milestone
      3.0.0
      Description
      @sanketverma1704
      sanketverma1704
      opened on Nov 22, 2022
      This came up during one of the conversations with @joshmoore. The idea is to extract functionalities not in V2 Spec and their respective tests from zarr-python. E.g. Zarr Specification V2 has the following stores:
      • Directory Store
      • Nested Directory Store
      • Zip file Store
      • Object Store
      • HTTP Store
      But in zarr-python, we have additional stores like SQLiteStore, MongoDBStore etc. If we could remove these additional stores from the zarr-python codebase and host them in a separate repo under zarr-developers, it'll help us:
      • Making the code run faster and more efficiently
      • Run fewer tests and faster completion of them
      • Reduce the noise from dependabot
      • Add maintainers for specific stores
      • Focus on a more concise codebase, thus helping us to add additional features when migrating from V2 to V3
      If a user/developer wishes to use the removed functionality, they can shoot pip install zarr-sqlitestore or similar.
      FYI: A similar repo is in the works for Numcodecs, which is taken care of by @joshmoore and @jakirkham.
      Please let me know your thoughts on this. Thanks!
      CC: @zarr-developers/python-core-devs
      Metadata
      Metadata
      Assignees
      No one assigned
        Labels
        help wantedIssue could use help from someone with familiarity on the topic
        Type
        No type
        Projects
        Milestone
        Relationships
        None yet
        Development
        No branches or pull requests
        Issue actions
        
, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

        Skip to content
        Thinning down zarr-python #1274
        Closed
        Closed
        Thinning down zarr-python#1274
        Labels
        help wantedIssue could use help from someone with familiarity on the topic
        Milestone
        3.0.0
        Description
        @sanketverma1704
        sanketverma1704
        opened on Nov 22, 2022
        This came up during one of the conversations with @joshmoore. The idea is to extract functionalities not in V2 Spec and their respective tests from zarr-python. E.g. Zarr Specification V2 has the following stores:
        • Directory Store
        • Nested Directory Store
        • Zip file Store
        • Object Store
        • HTTP Store
        But in zarr-python, we have additional stores like SQLiteStore, MongoDBStore etc. If we could remove these additional stores from the zarr-python codebase and host them in a separate repo under zarr-developers, it'll help us:
        • Making the code run faster and more efficiently
        • Run fewer tests and faster completion of them
        • Reduce the noise from dependabot
        • Add maintainers for specific stores
        • Focus on a more concise codebase, thus helping us to add additional features when migrating from V2 to V3
        If a user/developer wishes to use the removed functionality, they can shoot pip install zarr-sqlitestore or similar.
        FYI: A similar repo is in the works for Numcodecs, which is taken care of by @joshmoore and @jakirkham.
        Please let me know your thoughts on this. Thanks!
        CC: @zarr-developers/python-core-devs
        Metadata
        Metadata
        Assignees
        No one assigned
          Labels
          help wantedIssue could use help from someone with familiarity on the topic
          Type
          No type
          Projects
          Milestone
          Relationships
          None yet
          Development
          No branches or pull requests
          Issue actions
          
, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

          Skip to content
          Thinning down zarr-python #1274
          Closed
          Closed
          Thinning down zarr-python#1274
          Labels
          help wantedIssue could use help from someone with familiarity on the topic
          Milestone
          3.0.0
          Description
          @sanketverma1704
          sanketverma1704
          opened on Nov 22, 2022
          This came up during one of the conversations with @joshmoore. The idea is to extract functionalities not in V2 Spec and their respective tests from zarr-python. E.g. Zarr Specification V2 has the following stores:
          • Directory Store
          • Nested Directory Store
          • Zip file Store
          • Object Store
          • HTTP Store
          But in zarr-python, we have additional stores like SQLiteStore, MongoDBStore etc. If we could remove these additional stores from the zarr-python codebase and host them in a separate repo under zarr-developers, it'll help us:
          • Making the code run faster and more efficiently
          • Run fewer tests and faster completion of them
          • Reduce the noise from dependabot
          • Add maintainers for specific stores
          • Focus on a more concise codebase, thus helping us to add additional features when migrating from V2 to V3
          If a user/developer wishes to use the removed functionality, they can shoot pip install zarr-sqlitestore or similar.
          FYI: A similar repo is in the works for Numcodecs, which is taken care of by @joshmoore and @jakirkham.
          Please let me know your thoughts on this. Thanks!
          CC: @zarr-developers/python-core-devs
          Metadata
          Metadata
          Assignees
          No one assigned
            Labels
            help wantedIssue could use help from someone with familiarity on the topic
            Type
            No type
            Projects
            Milestone
            Relationships
            None yet
            Development
            No branches or pull requests
            Issue actions
            
, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

            Skip to content
            Thinning down zarr-python #1274
            Closed
            Closed
            Thinning down zarr-python#1274
            Labels
            help wantedIssue could use help from someone with familiarity on the topic
            Milestone
            3.0.0
            Description
            @sanketverma1704
            sanketverma1704
            opened on Nov 22, 2022
            This came up during one of the conversations with @joshmoore. The idea is to extract functionalities not in V2 Spec and their respective tests from zarr-python. E.g. Zarr Specification V2 has the following stores:
            • Directory Store
            • Nested Directory Store
            • Zip file Store
            • Object Store
            • HTTP Store
            But in zarr-python, we have additional stores like SQLiteStore, MongoDBStore etc. If we could remove these additional stores from the zarr-python codebase and host them in a separate repo under zarr-developers, it'll help us:
            • Making the code run faster and more efficiently
            • Run fewer tests and faster completion of them
            • Reduce the noise from dependabot
            • Add maintainers for specific stores
            • Focus on a more concise codebase, thus helping us to add additional features when migrating from V2 to V3
            If a user/developer wishes to use the removed functionality, they can shoot pip install zarr-sqlitestore or similar.
            FYI: A similar repo is in the works for Numcodecs, which is taken care of by @joshmoore and @jakirkham.
            Please let me know your thoughts on this. Thanks!
            CC: @zarr-developers/python-core-devs
            Metadata
            Metadata
            Assignees
            No one assigned
              Labels
              help wantedIssue could use help from someone with familiarity on the topic
              Type
              No type
              Projects
              Milestone
              Relationships
              None yet
              Development
              No branches or pull requests
              Issue actions
              
, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

              Skip to content
              Thinning down zarr-python #1274
              Closed
              Closed
              Thinning down zarr-python#1274
              Labels
              help wantedIssue could use help from someone with familiarity on the topic
              Milestone
              3.0.0
              Description
              @sanketverma1704
              sanketverma1704
              opened on Nov 22, 2022
              This came up during one of the conversations with @joshmoore. The idea is to extract functionalities not in V2 Spec and their respective tests from zarr-python. E.g. Zarr Specification V2 has the following stores:
              • Directory Store
              • Nested Directory Store
              • Zip file Store
              • Object Store
              • HTTP Store
              But in zarr-python, we have additional stores like SQLiteStore, MongoDBStore etc. If we could remove these additional stores from the zarr-python codebase and host them in a separate repo under zarr-developers, it'll help us:
              • Making the code run faster and more efficiently
              • Run fewer tests and faster completion of them
              • Reduce the noise from dependabot
              • Add maintainers for specific stores
              • Focus on a more concise codebase, thus helping us to add additional features when migrating from V2 to V3
              If a user/developer wishes to use the removed functionality, they can shoot pip install zarr-sqlitestore or similar.
              FYI: A similar repo is in the works for Numcodecs, which is taken care of by @joshmoore and @jakirkham.
              Please let me know your thoughts on this. Thanks!
              CC: @zarr-developers/python-core-devs
              Metadata
              Metadata
              Assignees
              No one assigned
                Labels
                help wantedIssue could use help from someone with familiarity on the topic
                Type
                No type
                Projects
                Milestone
                Relationships
                None yet
                Development
                No branches or pull requests
                Issue actions
                
, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

                Skip to content
                Thinning down zarr-python #1274
                Closed
                Closed
                Thinning down zarr-python#1274
                Labels
                help wantedIssue could use help from someone with familiarity on the topic
                Milestone
                3.0.0
                Description
                @sanketverma1704
                sanketverma1704
                opened on Nov 22, 2022
                This came up during one of the conversations with @joshmoore. The idea is to extract functionalities not in V2 Spec and their respective tests from zarr-python. E.g. Zarr Specification V2 has the following stores:
                • Directory Store
                • Nested Directory Store
                • Zip file Store
                • Object Store
                • HTTP Store
                But in zarr-python, we have additional stores like SQLiteStore, MongoDBStore etc. If we could remove these additional stores from the zarr-python codebase and host them in a separate repo under zarr-developers, it'll help us:
                • Making the code run faster and more efficiently
                • Run fewer tests and faster completion of them
                • Reduce the noise from dependabot
                • Add maintainers for specific stores
                • Focus on a more concise codebase, thus helping us to add additional features when migrating from V2 to V3
                If a user/developer wishes to use the removed functionality, they can shoot pip install zarr-sqlitestore or similar.
                FYI: A similar repo is in the works for Numcodecs, which is taken care of by @joshmoore and @jakirkham.
                Please let me know your thoughts on this. Thanks!
                CC: @zarr-developers/python-core-devs
                Metadata
                Metadata
                Assignees
                No one assigned
                  Labels
                  help wantedIssue could use help from someone with familiarity on the topic
                  Type
                  No type
                  Projects
                  Milestone
                  Relationships
                  None yet
                  Development
                  No branches or pull requests
                  Issue actions