Return ndarrays from encode and (where possible) decode - #136

Merged
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode
Nov 29, 2018
Merged

Return ndarrays from encode and (where possible) decode#136
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode

Conversation

@jakirkham

@jakirkhamjakirkham commented Nov 29, 2018

Copy link
Copy Markdown
Member

Ensures binary encode functions return ndarrays generally. This makes the output easier to anticipate and work with. Also allows more unusual outputs like objects' internal buffers to be exposed in a friendly way to end users (while avoiding copying). Where possible also ensure decode functions return ndarrays.

TODO:

  • Unit tests and/or doctests in docstrings
  • tox -e py37 passes locally
  • tox -e py27 passes locally
  • Docstrings and API docs for any new/modified user-facing classes and functions
  • Changes documented in docs/release.rst
  • tox -e docs passes locally
  • AppVeyor and Travis CI passes
  • Test coverage to 100% (Coveralls passes)

Standardize the output of the `encode` function a bit by ensuring it
returns an `ndarray`. This is done without copying as it only takes a
view onto the data. Has the advantage of leveraging objects under the
hood that might not appear as nice like objects' internal buffers
without impacting the user or unnecessarily copying the data to `bytes`.
Also makes it a bit easier to work with the output as users can do
anything they would normally want to with `ndarray`s.
Where possible, ensure that `decode` returns an `ndarray`. Many codecs
already do this except for some unusual ones that work with Python
objects or text data. In the cases where the data is known to be binary,
try to return `ndarray`s. This makes it pretty easy for end users of
this data to do whatever they would like with it.
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Please let me know how you want this listed in the changelog (if at all).

@alimanfooalimanfoo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jakirkham, LGTM. Re changelog, I think it would be good to add something, how about something like:

* Return values from encode() and decode() methods are now returned as numpy arrays for consistency across codecs. By :user:`John Kirkham <jakirkham>`, :issue:`136`.

@alimanfoo

Copy link
Copy Markdown
Member

Went ahead and added release notes and resolved merge conflict, hope that's OK. Will merge if CI passes.

@jakirkhamjakirkham added this to the 0.6.0 milestone Nov 29, 2018
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Great, thanks for doing that, @alimanfoo. SGTM

@alimanfoo
alimanfoo merged commit fd46c4c into zarr-developers:masterNov 29, 2018
@jakirkham
jakirkham deleted the ret_ndarray_encode_decode branch November 29, 2018 22:03
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Apologies for not opening a new issue for this. Happy to do so if needed.

Should we cut the release now or were there some other things still needed?

@alimanfoo

Copy link
Copy Markdown
Member

No problem at all. I think cut the release now: #137.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@jakirkham@alimanfoo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

Return ndarrays from encode and (where possible) decode - #136

Merged
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode
Nov 29, 2018
Merged

Return ndarrays from encode and (where possible) decode#136
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode

Conversation

@jakirkham

@jakirkhamjakirkham commented Nov 29, 2018

Copy link
Copy Markdown
Member

Ensures binary encode functions return ndarrays generally. This makes the output easier to anticipate and work with. Also allows more unusual outputs like objects' internal buffers to be exposed in a friendly way to end users (while avoiding copying). Where possible also ensure decode functions return ndarrays.

TODO:

  • Unit tests and/or doctests in docstrings
  • tox -e py37 passes locally
  • tox -e py27 passes locally
  • Docstrings and API docs for any new/modified user-facing classes and functions
  • Changes documented in docs/release.rst
  • tox -e docs passes locally
  • AppVeyor and Travis CI passes
  • Test coverage to 100% (Coveralls passes)

Standardize the output of the `encode` function a bit by ensuring it
returns an `ndarray`. This is done without copying as it only takes a
view onto the data. Has the advantage of leveraging objects under the
hood that might not appear as nice like objects' internal buffers
without impacting the user or unnecessarily copying the data to `bytes`.
Also makes it a bit easier to work with the output as users can do
anything they would normally want to with `ndarray`s.
Where possible, ensure that `decode` returns an `ndarray`. Many codecs
already do this except for some unusual ones that work with Python
objects or text data. In the cases where the data is known to be binary,
try to return `ndarray`s. This makes it pretty easy for end users of
this data to do whatever they would like with it.
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Please let me know how you want this listed in the changelog (if at all).

@alimanfooalimanfoo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jakirkham, LGTM. Re changelog, I think it would be good to add something, how about something like:

* Return values from encode() and decode() methods are now returned as numpy arrays for consistency across codecs. By :user:`John Kirkham <jakirkham>`, :issue:`136`.

@alimanfoo

Copy link
Copy Markdown
Member

Went ahead and added release notes and resolved merge conflict, hope that's OK. Will merge if CI passes.

@jakirkhamjakirkham added this to the 0.6.0 milestone Nov 29, 2018
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Great, thanks for doing that, @alimanfoo. SGTM

@alimanfoo
alimanfoo merged commit fd46c4c into zarr-developers:masterNov 29, 2018
@jakirkham
jakirkham deleted the ret_ndarray_encode_decode branch November 29, 2018 22:03
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Apologies for not opening a new issue for this. Happy to do so if needed.

Should we cut the release now or were there some other things still needed?

@alimanfoo

Copy link
Copy Markdown
Member

No problem at all. I think cut the release now: #137.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@jakirkham@alimanfoo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Return ndarrays from encode and (where possible) decode - #136

Merged
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode
Nov 29, 2018
Merged

Return ndarrays from encode and (where possible) decode#136
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode

Conversation

@jakirkham

@jakirkhamjakirkham commented Nov 29, 2018

Copy link
Copy Markdown
Member

Ensures binary encode functions return ndarrays generally. This makes the output easier to anticipate and work with. Also allows more unusual outputs like objects' internal buffers to be exposed in a friendly way to end users (while avoiding copying). Where possible also ensure decode functions return ndarrays.

TODO:

  • Unit tests and/or doctests in docstrings
  • tox -e py37 passes locally
  • tox -e py27 passes locally
  • Docstrings and API docs for any new/modified user-facing classes and functions
  • Changes documented in docs/release.rst
  • tox -e docs passes locally
  • AppVeyor and Travis CI passes
  • Test coverage to 100% (Coveralls passes)

Standardize the output of the `encode` function a bit by ensuring it
returns an `ndarray`. This is done without copying as it only takes a
view onto the data. Has the advantage of leveraging objects under the
hood that might not appear as nice like objects' internal buffers
without impacting the user or unnecessarily copying the data to `bytes`.
Also makes it a bit easier to work with the output as users can do
anything they would normally want to with `ndarray`s.
Where possible, ensure that `decode` returns an `ndarray`. Many codecs
already do this except for some unusual ones that work with Python
objects or text data. In the cases where the data is known to be binary,
try to return `ndarray`s. This makes it pretty easy for end users of
this data to do whatever they would like with it.
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Please let me know how you want this listed in the changelog (if at all).

@alimanfooalimanfoo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jakirkham, LGTM. Re changelog, I think it would be good to add something, how about something like:

* Return values from encode() and decode() methods are now returned as numpy arrays for consistency across codecs. By :user:`John Kirkham <jakirkham>`, :issue:`136`.

@alimanfoo

Copy link
Copy Markdown
Member

Went ahead and added release notes and resolved merge conflict, hope that's OK. Will merge if CI passes.

@jakirkhamjakirkham added this to the 0.6.0 milestone Nov 29, 2018
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Great, thanks for doing that, @alimanfoo. SGTM

@alimanfoo
alimanfoo merged commit fd46c4c into zarr-developers:masterNov 29, 2018
@jakirkham
jakirkham deleted the ret_ndarray_encode_decode branch November 29, 2018 22:03
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Apologies for not opening a new issue for this. Happy to do so if needed.

Should we cut the release now or were there some other things still needed?

@alimanfoo

Copy link
Copy Markdown
Member

No problem at all. I think cut the release now: #137.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@jakirkham@alimanfoo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Return ndarrays from encode and (where possible) decode - #136

Merged
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode
Nov 29, 2018
Merged

Return ndarrays from encode and (where possible) decode#136
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode

Conversation

@jakirkham

@jakirkhamjakirkham commented Nov 29, 2018

Copy link
Copy Markdown
Member

Ensures binary encode functions return ndarrays generally. This makes the output easier to anticipate and work with. Also allows more unusual outputs like objects' internal buffers to be exposed in a friendly way to end users (while avoiding copying). Where possible also ensure decode functions return ndarrays.

TODO:

  • Unit tests and/or doctests in docstrings
  • tox -e py37 passes locally
  • tox -e py27 passes locally
  • Docstrings and API docs for any new/modified user-facing classes and functions
  • Changes documented in docs/release.rst
  • tox -e docs passes locally
  • AppVeyor and Travis CI passes
  • Test coverage to 100% (Coveralls passes)

Standardize the output of the `encode` function a bit by ensuring it
returns an `ndarray`. This is done without copying as it only takes a
view onto the data. Has the advantage of leveraging objects under the
hood that might not appear as nice like objects' internal buffers
without impacting the user or unnecessarily copying the data to `bytes`.
Also makes it a bit easier to work with the output as users can do
anything they would normally want to with `ndarray`s.
Where possible, ensure that `decode` returns an `ndarray`. Many codecs
already do this except for some unusual ones that work with Python
objects or text data. In the cases where the data is known to be binary,
try to return `ndarray`s. This makes it pretty easy for end users of
this data to do whatever they would like with it.
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Please let me know how you want this listed in the changelog (if at all).

@alimanfooalimanfoo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jakirkham, LGTM. Re changelog, I think it would be good to add something, how about something like:

* Return values from encode() and decode() methods are now returned as numpy arrays for consistency across codecs. By :user:`John Kirkham <jakirkham>`, :issue:`136`.

@alimanfoo

Copy link
Copy Markdown
Member

Went ahead and added release notes and resolved merge conflict, hope that's OK. Will merge if CI passes.

@jakirkhamjakirkham added this to the 0.6.0 milestone Nov 29, 2018
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Great, thanks for doing that, @alimanfoo. SGTM

@alimanfoo
alimanfoo merged commit fd46c4c into zarr-developers:masterNov 29, 2018
@jakirkham
jakirkham deleted the ret_ndarray_encode_decode branch November 29, 2018 22:03
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Apologies for not opening a new issue for this. Happy to do so if needed.

Should we cut the release now or were there some other things still needed?

@alimanfoo

Copy link
Copy Markdown
Member

No problem at all. I think cut the release now: #137.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@jakirkham@alimanfoo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

Return ndarrays from encode and (where possible) decode - #136

Merged
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode
Nov 29, 2018
Merged

Return ndarrays from encode and (where possible) decode#136
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode

Conversation

@jakirkham

@jakirkhamjakirkham commented Nov 29, 2018

Copy link
Copy Markdown
Member

Ensures binary encode functions return ndarrays generally. This makes the output easier to anticipate and work with. Also allows more unusual outputs like objects' internal buffers to be exposed in a friendly way to end users (while avoiding copying). Where possible also ensure decode functions return ndarrays.

TODO:

  • Unit tests and/or doctests in docstrings
  • tox -e py37 passes locally
  • tox -e py27 passes locally
  • Docstrings and API docs for any new/modified user-facing classes and functions
  • Changes documented in docs/release.rst
  • tox -e docs passes locally
  • AppVeyor and Travis CI passes
  • Test coverage to 100% (Coveralls passes)

Standardize the output of the `encode` function a bit by ensuring it
returns an `ndarray`. This is done without copying as it only takes a
view onto the data. Has the advantage of leveraging objects under the
hood that might not appear as nice like objects' internal buffers
without impacting the user or unnecessarily copying the data to `bytes`.
Also makes it a bit easier to work with the output as users can do
anything they would normally want to with `ndarray`s.
Where possible, ensure that `decode` returns an `ndarray`. Many codecs
already do this except for some unusual ones that work with Python
objects or text data. In the cases where the data is known to be binary,
try to return `ndarray`s. This makes it pretty easy for end users of
this data to do whatever they would like with it.
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Please let me know how you want this listed in the changelog (if at all).

@alimanfooalimanfoo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jakirkham, LGTM. Re changelog, I think it would be good to add something, how about something like:

* Return values from encode() and decode() methods are now returned as numpy arrays for consistency across codecs. By :user:`John Kirkham <jakirkham>`, :issue:`136`.

@alimanfoo

Copy link
Copy Markdown
Member

Went ahead and added release notes and resolved merge conflict, hope that's OK. Will merge if CI passes.

@jakirkhamjakirkham added this to the 0.6.0 milestone Nov 29, 2018
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Great, thanks for doing that, @alimanfoo. SGTM

@alimanfoo
alimanfoo merged commit fd46c4c into zarr-developers:masterNov 29, 2018
@jakirkham
jakirkham deleted the ret_ndarray_encode_decode branch November 29, 2018 22:03
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Apologies for not opening a new issue for this. Happy to do so if needed.

Should we cut the release now or were there some other things still needed?

@alimanfoo

Copy link
Copy Markdown
Member

No problem at all. I think cut the release now: #137.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@jakirkham@alimanfoo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Return ndarrays from encode and (where possible) decode - #136

Merged
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode
Nov 29, 2018
Merged

Return ndarrays from encode and (where possible) decode#136
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode

Conversation

@jakirkham

@jakirkhamjakirkham commented Nov 29, 2018

Copy link
Copy Markdown
Member

Ensures binary encode functions return ndarrays generally. This makes the output easier to anticipate and work with. Also allows more unusual outputs like objects' internal buffers to be exposed in a friendly way to end users (while avoiding copying). Where possible also ensure decode functions return ndarrays.

TODO:

  • Unit tests and/or doctests in docstrings
  • tox -e py37 passes locally
  • tox -e py27 passes locally
  • Docstrings and API docs for any new/modified user-facing classes and functions
  • Changes documented in docs/release.rst
  • tox -e docs passes locally
  • AppVeyor and Travis CI passes
  • Test coverage to 100% (Coveralls passes)

Standardize the output of the `encode` function a bit by ensuring it
returns an `ndarray`. This is done without copying as it only takes a
view onto the data. Has the advantage of leveraging objects under the
hood that might not appear as nice like objects' internal buffers
without impacting the user or unnecessarily copying the data to `bytes`.
Also makes it a bit easier to work with the output as users can do
anything they would normally want to with `ndarray`s.
Where possible, ensure that `decode` returns an `ndarray`. Many codecs
already do this except for some unusual ones that work with Python
objects or text data. In the cases where the data is known to be binary,
try to return `ndarray`s. This makes it pretty easy for end users of
this data to do whatever they would like with it.
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Please let me know how you want this listed in the changelog (if at all).

@alimanfooalimanfoo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jakirkham, LGTM. Re changelog, I think it would be good to add something, how about something like:

* Return values from encode() and decode() methods are now returned as numpy arrays for consistency across codecs. By :user:`John Kirkham <jakirkham>`, :issue:`136`.

@alimanfoo

Copy link
Copy Markdown
Member

Went ahead and added release notes and resolved merge conflict, hope that's OK. Will merge if CI passes.

@jakirkhamjakirkham added this to the 0.6.0 milestone Nov 29, 2018
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Great, thanks for doing that, @alimanfoo. SGTM

@alimanfoo
alimanfoo merged commit fd46c4c into zarr-developers:masterNov 29, 2018
@jakirkham
jakirkham deleted the ret_ndarray_encode_decode branch November 29, 2018 22:03
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Apologies for not opening a new issue for this. Happy to do so if needed.

Should we cut the release now or were there some other things still needed?

@alimanfoo

Copy link
Copy Markdown
Member

No problem at all. I think cut the release now: #137.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@jakirkham@alimanfoo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

Return ndarrays from encode and (where possible) decode - #136

Merged
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode
Nov 29, 2018
Merged

Return ndarrays from encode and (where possible) decode#136
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode

Conversation

@jakirkham

@jakirkhamjakirkham commented Nov 29, 2018

Copy link
Copy Markdown
Member

Ensures binary encode functions return ndarrays generally. This makes the output easier to anticipate and work with. Also allows more unusual outputs like objects' internal buffers to be exposed in a friendly way to end users (while avoiding copying). Where possible also ensure decode functions return ndarrays.

TODO:

  • Unit tests and/or doctests in docstrings
  • tox -e py37 passes locally
  • tox -e py27 passes locally
  • Docstrings and API docs for any new/modified user-facing classes and functions
  • Changes documented in docs/release.rst
  • tox -e docs passes locally
  • AppVeyor and Travis CI passes
  • Test coverage to 100% (Coveralls passes)

Standardize the output of the `encode` function a bit by ensuring it
returns an `ndarray`. This is done without copying as it only takes a
view onto the data. Has the advantage of leveraging objects under the
hood that might not appear as nice like objects' internal buffers
without impacting the user or unnecessarily copying the data to `bytes`.
Also makes it a bit easier to work with the output as users can do
anything they would normally want to with `ndarray`s.
Where possible, ensure that `decode` returns an `ndarray`. Many codecs
already do this except for some unusual ones that work with Python
objects or text data. In the cases where the data is known to be binary,
try to return `ndarray`s. This makes it pretty easy for end users of
this data to do whatever they would like with it.
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Please let me know how you want this listed in the changelog (if at all).

@alimanfooalimanfoo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jakirkham, LGTM. Re changelog, I think it would be good to add something, how about something like:

* Return values from encode() and decode() methods are now returned as numpy arrays for consistency across codecs. By :user:`John Kirkham <jakirkham>`, :issue:`136`.

@alimanfoo

Copy link
Copy Markdown
Member

Went ahead and added release notes and resolved merge conflict, hope that's OK. Will merge if CI passes.

@jakirkhamjakirkham added this to the 0.6.0 milestone Nov 29, 2018
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Great, thanks for doing that, @alimanfoo. SGTM

@alimanfoo
alimanfoo merged commit fd46c4c into zarr-developers:masterNov 29, 2018
@jakirkham
jakirkham deleted the ret_ndarray_encode_decode branch November 29, 2018 22:03
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Apologies for not opening a new issue for this. Happy to do so if needed.

Should we cut the release now or were there some other things still needed?

@alimanfoo

Copy link
Copy Markdown
Member

No problem at all. I think cut the release now: #137.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@jakirkham@alimanfoo
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

Return ndarrays from encode and (where possible) decode - #136

Merged
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode
Nov 29, 2018
Merged

Return ndarrays from encode and (where possible) decode#136
alimanfoo merged 5 commits into
zarr-developers:masterfrom
jakirkham:ret_ndarray_encode_decode

Conversation

@jakirkham

@jakirkhamjakirkham commented Nov 29, 2018

Copy link
Copy Markdown
Member

Ensures binary encode functions return ndarrays generally. This makes the output easier to anticipate and work with. Also allows more unusual outputs like objects' internal buffers to be exposed in a friendly way to end users (while avoiding copying). Where possible also ensure decode functions return ndarrays.

TODO:

  • Unit tests and/or doctests in docstrings
  • tox -e py37 passes locally
  • tox -e py27 passes locally
  • Docstrings and API docs for any new/modified user-facing classes and functions
  • Changes documented in docs/release.rst
  • tox -e docs passes locally
  • AppVeyor and Travis CI passes
  • Test coverage to 100% (Coveralls passes)

Standardize the output of the `encode` function a bit by ensuring it
returns an `ndarray`. This is done without copying as it only takes a
view onto the data. Has the advantage of leveraging objects under the
hood that might not appear as nice like objects' internal buffers
without impacting the user or unnecessarily copying the data to `bytes`.
Also makes it a bit easier to work with the output as users can do
anything they would normally want to with `ndarray`s.
Where possible, ensure that `decode` returns an `ndarray`. Many codecs
already do this except for some unusual ones that work with Python
objects or text data. In the cases where the data is known to be binary,
try to return `ndarray`s. This makes it pretty easy for end users of
this data to do whatever they would like with it.
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Please let me know how you want this listed in the changelog (if at all).

@alimanfooalimanfoo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jakirkham, LGTM. Re changelog, I think it would be good to add something, how about something like:

* Return values from encode() and decode() methods are now returned as numpy arrays for consistency across codecs. By :user:`John Kirkham <jakirkham>`, :issue:`136`.

@alimanfoo

Copy link
Copy Markdown
Member

Went ahead and added release notes and resolved merge conflict, hope that's OK. Will merge if CI passes.

@jakirkhamjakirkham added this to the 0.6.0 milestone Nov 29, 2018
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Great, thanks for doing that, @alimanfoo. SGTM

@alimanfoo
alimanfoo merged commit fd46c4c into zarr-developers:masterNov 29, 2018
@jakirkham
jakirkham deleted the ret_ndarray_encode_decode branch November 29, 2018 22:03
@jakirkham

Copy link
Copy Markdown
MemberAuthor

Apologies for not opening a new issue for this. Happy to do so if needed.

Should we cut the release now or were there some other things still needed?

@alimanfoo

Copy link
Copy Markdown
Member

No problem at all. I think cut the release now: #137.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@jakirkham@alimanfoo