ARROW-9465: [Python] Improve ergonomics of compute module - #8163

Closed
pitrou wants to merge 4 commits into
apache:masterfrom
pitrou:ARROW-9465-py-compute-cosmetics
Closed

ARROW-9465: [Python] Improve ergonomics of compute module#8163
pitrou wants to merge 4 commits into
apache:masterfrom
pitrou:ARROW-9465-py-compute-cosmetics

Conversation

@pitrou

@pitroupitrou commented Sep 10, 2020

Copy link
Copy Markdown
Member

Lots of assorted things here:

  • Automatically generate global wrappers for calling
    registered compute functions
  • Improve metadata of such wrappers (e.g. name, docstring)
  • Make it easier to pass options (for example via kwargs)
  • Type-check options
  • Add some docstrings
  • Expose more function attributes (e.g. arity)
  • Fix some crashes

@github-actions

github-actionsBot commented Sep 10, 2020

Copy link
Copy Markdown

https://issues.apache.org/jira/browse/ARROW-9465

@arw2019arw2019 mentioned this pull request Sep 13, 2020
Closed
jorisvandenbossche
jorisvandenbossche reviewed Sep 14, 2020
View reviewed changes

@jorisvandenbosschejorisvandenbossche left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvements!

Comment threadpython/pyarrow/_compute.pyx Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should "meta" be explained here as well?

@pitroupitrouSep 14, 2020

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you're right.

Comment threadcpp/src/arrow/compute/api_aggregate.h Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we simply change this? (is it public on the C++ side?) And it is changed for consistency with how it's called elsewhere?

@pitroupitrouSep 14, 2020

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The next version will be 2.0.0, meaning it's ok to break APIs. Of course, we should try to minimize breakage, but this one aims to fix an inconsistency.

@pitrou
ARROW-9465: [Python] Improve ergonomics of compute module
57ccea0
Lots of assorted things here:
* Automatically generate global wrappers for calling
registered compute functions
* Improve metadata of such wrappers (e.g. name, docstring)
* Make it easier to pass options (for example via kwargs)
* Type-check options
* Add some docstrings
* Expose more function attributes (e.g. arity)
* Fix some crashes
@pitrou
Add automatic scalar conversion
6e82d7b
@pitrou
Address review comment
648ee85
@pitrou
pitrouforce-pushed the ARROW-9465-py-compute-cosmetics branch from 7cce626 to 648ee85CompareSeptember 14, 2020 13:35
@pitrou

pitrou commented Sep 14, 2020

Copy link
Copy Markdown
MemberAuthor

CI failures here look unrelated (though someone should really take a look at them).

@pitrou
pitrou deleted the ARROW-9465-py-compute-cosmetics branch September 14, 2020 17:22
@maartenbreddelsmaartenbreddels mentioned this pull request Oct 6, 2020
Closed
pitrou added a commit that referenced this pull request Oct 8, 2020
@arw2019@pitrou
ARROW-9967: [Python] Add compute module docs + expose more option cla…
ba7ee65
…sses
#8163 exposes `pyarrow.compute` kernels and generates their docstrings. This PR adds documentation for the module in the User Guide and the Python API reference.
Closes#8145 from arw2019/ARROW-7871
Lead-authored-by: arw2019 <andrew.r.wieteska@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@asfimportasfimport mentioned this pull request Sep 14, 2020
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@pitrou@jorisvandenbossche
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
ARROW-9465: [Python] Improve ergonomics of compute module - #8163
Closed
pitrou wants to merge 4 commits into
apache:masterfrom 
pitrou:ARROW-9465-py-compute-cosmetics
Closed
ARROW-9465: [Python] Improve ergonomics of compute module#8163
pitrou wants to merge 4 commits into
apache:masterfrom 
pitrou:ARROW-9465-py-compute-cosmetics
Conversation
@pitrou
@pitroupitrou
commented
Sep 10, 2020

Copy link


Copy Markdown

Member
Lots of assorted things here:
  • Automatically generate global wrappers for calling
    
registered compute functions
  • Improve metadata of such wrappers (e.g. name, docstring)
  • Make it easier to pass options (for example via kwargs)
  • Type-check options
  • Add some docstrings
  • Expose more function attributes (e.g. arity)
  • Fix some crashes
@github-actions
github-actionsBot
commented
Sep 10, 2020

Copy link


Copy Markdown

https://issues.apache.org/jira/browse/ARROW-9465
@arw2019arw2019
mentioned this pull request
Sep 13, 2020
 Closed

jorisvandenbossche
jorisvandenbossche
reviewed
Sep 14, 2020
View reviewed changes
@jorisvandenbosschejorisvandenbossche
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvements!
Comment threadpython/pyarrow/_compute.pyx
Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should "meta" be explained here as well?
@pitroupitrouSep 14, 2020

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you're right.
Comment threadcpp/src/arrow/compute/api_aggregate.h
Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we simply change this? (is it public on the C++ side?) And it is changed for consistency with how it's called elsewhere?
@pitroupitrouSep 14, 2020

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The next version will be 2.0.0, meaning it's ok to break APIs. Of course, we should try to minimize breakage, but this one aims to fix an inconsistency.
@pitrou
ARROW-9465: [Python] Improve ergonomics of compute module
57ccea0
Lots of assorted things here:
* Automatically generate global wrappers for calling
registered compute functions
* Improve metadata of such wrappers (e.g. name, docstring)
* Make it easier to pass options (for example via kwargs)
* Type-check options
* Add some docstrings
* Expose more function attributes (e.g. arity)
* Fix some crashes
@pitrou
Add automatic scalar conversion
6e82d7b
@pitrou
Address review comment
648ee85
@pitrou
pitrouforce-pushed
the
ARROW-9465-py-compute-cosmetics
branch
from
7cce626 to
648ee85CompareSeptember 14, 2020 13:35
@pitrou
pitrou
commented
Sep 14, 2020

Copy link


Copy Markdown

MemberAuthor
CI failures here look unrelated (though someone should really take a look at them).
@pitrou
pitrou
deleted the

ARROW-9465-py-compute-cosmetics

branch
September 14, 2020 17:22
@maartenbreddelsmaartenbreddels
mentioned this pull request
Oct 6, 2020
 Closed

pitrou added a commit
that referenced
this pull request
Oct 8, 2020
@arw2019@pitrou
ARROW-9967: [Python] Add compute module docs + expose more option cla…
ba7ee65
…sses
#8163 exposes `pyarrow.compute` kernels and generates their docstrings. This PR adds documentation for the module in the User Guide and the Python API reference.
Closes#8145 from arw2019/ARROW-7871
Lead-authored-by: arw2019 <andrew.r.wieteska@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@asfimportasfimport
mentioned this pull request
Sep 14, 2020

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@pitrou@jorisvandenbossche

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-9465: [Python] Improve ergonomics of compute module - #8163
Closed
pitrou wants to merge 4 commits into
apache:masterfrom 
pitrou:ARROW-9465-py-compute-cosmetics
Closed
ARROW-9465: [Python] Improve ergonomics of compute module#8163
pitrou wants to merge 4 commits into
apache:masterfrom 
pitrou:ARROW-9465-py-compute-cosmetics
Conversation
@pitrou
@pitroupitrou
commented
Sep 10, 2020

Copy link


Copy Markdown

Member
Lots of assorted things here:
  • Automatically generate global wrappers for calling
    
registered compute functions
  • Improve metadata of such wrappers (e.g. name, docstring)
  • Make it easier to pass options (for example via kwargs)
  • Type-check options
  • Add some docstrings
  • Expose more function attributes (e.g. arity)
  • Fix some crashes
@github-actions
github-actionsBot
commented
Sep 10, 2020

Copy link


Copy Markdown

https://issues.apache.org/jira/browse/ARROW-9465
@arw2019arw2019
mentioned this pull request
Sep 13, 2020
 Closed

jorisvandenbossche
jorisvandenbossche
reviewed
Sep 14, 2020
View reviewed changes
@jorisvandenbosschejorisvandenbossche
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvements!
Comment threadpython/pyarrow/_compute.pyx
Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should "meta" be explained here as well?
@pitroupitrouSep 14, 2020

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you're right.
Comment threadcpp/src/arrow/compute/api_aggregate.h
Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we simply change this? (is it public on the C++ side?) And it is changed for consistency with how it's called elsewhere?
@pitroupitrouSep 14, 2020

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The next version will be 2.0.0, meaning it's ok to break APIs. Of course, we should try to minimize breakage, but this one aims to fix an inconsistency.
@pitrou
ARROW-9465: [Python] Improve ergonomics of compute module
57ccea0
Lots of assorted things here:
* Automatically generate global wrappers for calling
registered compute functions
* Improve metadata of such wrappers (e.g. name, docstring)
* Make it easier to pass options (for example via kwargs)
* Type-check options
* Add some docstrings
* Expose more function attributes (e.g. arity)
* Fix some crashes
@pitrou
Add automatic scalar conversion
6e82d7b
@pitrou
Address review comment
648ee85
@pitrou
pitrouforce-pushed
the
ARROW-9465-py-compute-cosmetics
branch
from
7cce626 to
648ee85CompareSeptember 14, 2020 13:35
@pitrou
pitrou
commented
Sep 14, 2020

Copy link


Copy Markdown

MemberAuthor
CI failures here look unrelated (though someone should really take a look at them).
@pitrou
pitrou
deleted the

ARROW-9465-py-compute-cosmetics

branch
September 14, 2020 17:22
@maartenbreddelsmaartenbreddels
mentioned this pull request
Oct 6, 2020
 Closed

pitrou added a commit
that referenced
this pull request
Oct 8, 2020
@arw2019@pitrou
ARROW-9967: [Python] Add compute module docs + expose more option cla…
ba7ee65
…sses
#8163 exposes `pyarrow.compute` kernels and generates their docstrings. This PR adds documentation for the module in the User Guide and the Python API reference.
Closes#8145 from arw2019/ARROW-7871
Lead-authored-by: arw2019 <andrew.r.wieteska@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@asfimportasfimport
mentioned this pull request
Sep 14, 2020

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@pitrou@jorisvandenbossche

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-9465: [Python] Improve ergonomics of compute module - #8163
Closed
pitrou wants to merge 4 commits into
apache:masterfrom 
pitrou:ARROW-9465-py-compute-cosmetics
Closed
ARROW-9465: [Python] Improve ergonomics of compute module#8163
pitrou wants to merge 4 commits into
apache:masterfrom 
pitrou:ARROW-9465-py-compute-cosmetics
Conversation
@pitrou
@pitroupitrou
commented
Sep 10, 2020

Copy link


Copy Markdown

Member
Lots of assorted things here:
  • Automatically generate global wrappers for calling
    
registered compute functions
  • Improve metadata of such wrappers (e.g. name, docstring)
  • Make it easier to pass options (for example via kwargs)
  • Type-check options
  • Add some docstrings
  • Expose more function attributes (e.g. arity)
  • Fix some crashes
@github-actions
github-actionsBot
commented
Sep 10, 2020

Copy link


Copy Markdown

https://issues.apache.org/jira/browse/ARROW-9465
@arw2019arw2019
mentioned this pull request
Sep 13, 2020
 Closed

jorisvandenbossche
jorisvandenbossche
reviewed
Sep 14, 2020
View reviewed changes
@jorisvandenbosschejorisvandenbossche
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvements!
Comment threadpython/pyarrow/_compute.pyx
Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should "meta" be explained here as well?
@pitroupitrouSep 14, 2020

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you're right.
Comment threadcpp/src/arrow/compute/api_aggregate.h
Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we simply change this? (is it public on the C++ side?) And it is changed for consistency with how it's called elsewhere?
@pitroupitrouSep 14, 2020

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The next version will be 2.0.0, meaning it's ok to break APIs. Of course, we should try to minimize breakage, but this one aims to fix an inconsistency.
@pitrou
ARROW-9465: [Python] Improve ergonomics of compute module
57ccea0
Lots of assorted things here:
* Automatically generate global wrappers for calling
registered compute functions
* Improve metadata of such wrappers (e.g. name, docstring)
* Make it easier to pass options (for example via kwargs)
* Type-check options
* Add some docstrings
* Expose more function attributes (e.g. arity)
* Fix some crashes
@pitrou
Add automatic scalar conversion
6e82d7b
@pitrou
Address review comment
648ee85
@pitrou
pitrouforce-pushed
the
ARROW-9465-py-compute-cosmetics
branch
from
7cce626 to
648ee85CompareSeptember 14, 2020 13:35
@pitrou
pitrou
commented
Sep 14, 2020

Copy link


Copy Markdown

MemberAuthor
CI failures here look unrelated (though someone should really take a look at them).
@pitrou
pitrou
deleted the

ARROW-9465-py-compute-cosmetics

branch
September 14, 2020 17:22
@maartenbreddelsmaartenbreddels
mentioned this pull request
Oct 6, 2020
 Closed

pitrou added a commit
that referenced
this pull request
Oct 8, 2020
@arw2019@pitrou
ARROW-9967: [Python] Add compute module docs + expose more option cla…
ba7ee65
…sses
#8163 exposes `pyarrow.compute` kernels and generates their docstrings. This PR adds documentation for the module in the User Guide and the Python API reference.
Closes#8145 from arw2019/ARROW-7871
Lead-authored-by: arw2019 <andrew.r.wieteska@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@asfimportasfimport
mentioned this pull request
Sep 14, 2020

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@pitrou@jorisvandenbossche

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
ARROW-9465: [Python] Improve ergonomics of compute module - #8163
Closed
pitrou wants to merge 4 commits into
apache:masterfrom 
pitrou:ARROW-9465-py-compute-cosmetics
Closed
ARROW-9465: [Python] Improve ergonomics of compute module#8163
pitrou wants to merge 4 commits into
apache:masterfrom 
pitrou:ARROW-9465-py-compute-cosmetics
Conversation
@pitrou
@pitroupitrou
commented
Sep 10, 2020

Copy link


Copy Markdown

Member
Lots of assorted things here:
  • Automatically generate global wrappers for calling
    
registered compute functions
  • Improve metadata of such wrappers (e.g. name, docstring)
  • Make it easier to pass options (for example via kwargs)
  • Type-check options
  • Add some docstrings
  • Expose more function attributes (e.g. arity)
  • Fix some crashes
@github-actions
github-actionsBot
commented
Sep 10, 2020

Copy link


Copy Markdown

https://issues.apache.org/jira/browse/ARROW-9465
@arw2019arw2019
mentioned this pull request
Sep 13, 2020
 Closed

jorisvandenbossche
jorisvandenbossche
reviewed
Sep 14, 2020
View reviewed changes
@jorisvandenbosschejorisvandenbossche
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvements!
Comment threadpython/pyarrow/_compute.pyx
Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should "meta" be explained here as well?
@pitroupitrouSep 14, 2020

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you're right.
Comment threadcpp/src/arrow/compute/api_aggregate.h
Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we simply change this? (is it public on the C++ side?) And it is changed for consistency with how it's called elsewhere?
@pitroupitrouSep 14, 2020

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The next version will be 2.0.0, meaning it's ok to break APIs. Of course, we should try to minimize breakage, but this one aims to fix an inconsistency.
@pitrou
ARROW-9465: [Python] Improve ergonomics of compute module
57ccea0
Lots of assorted things here:
* Automatically generate global wrappers for calling
registered compute functions
* Improve metadata of such wrappers (e.g. name, docstring)
* Make it easier to pass options (for example via kwargs)
* Type-check options
* Add some docstrings
* Expose more function attributes (e.g. arity)
* Fix some crashes
@pitrou
Add automatic scalar conversion
6e82d7b
@pitrou
Address review comment
648ee85
@pitrou
pitrouforce-pushed
the
ARROW-9465-py-compute-cosmetics
branch
from
7cce626 to
648ee85CompareSeptember 14, 2020 13:35
@pitrou
pitrou
commented
Sep 14, 2020

Copy link


Copy Markdown

MemberAuthor
CI failures here look unrelated (though someone should really take a look at them).
@pitrou
pitrou
deleted the

ARROW-9465-py-compute-cosmetics

branch
September 14, 2020 17:22
@maartenbreddelsmaartenbreddels
mentioned this pull request
Oct 6, 2020
 Closed

pitrou added a commit
that referenced
this pull request
Oct 8, 2020
@arw2019@pitrou
ARROW-9967: [Python] Add compute module docs + expose more option cla…
ba7ee65
…sses
#8163 exposes `pyarrow.compute` kernels and generates their docstrings. This PR adds documentation for the module in the User Guide and the Python API reference.
Closes#8145 from arw2019/ARROW-7871
Lead-authored-by: arw2019 <andrew.r.wieteska@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@asfimportasfimport
mentioned this pull request
Sep 14, 2020

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@pitrou@jorisvandenbossche

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-9465: [Python] Improve ergonomics of compute module - #8163
Closed
pitrou wants to merge 4 commits into
apache:masterfrom 
pitrou:ARROW-9465-py-compute-cosmetics
Closed
ARROW-9465: [Python] Improve ergonomics of compute module#8163
pitrou wants to merge 4 commits into
apache:masterfrom 
pitrou:ARROW-9465-py-compute-cosmetics
Conversation
@pitrou
@pitroupitrou
commented
Sep 10, 2020

Copy link


Copy Markdown

Member
Lots of assorted things here:
  • Automatically generate global wrappers for calling
    
registered compute functions
  • Improve metadata of such wrappers (e.g. name, docstring)
  • Make it easier to pass options (for example via kwargs)
  • Type-check options
  • Add some docstrings
  • Expose more function attributes (e.g. arity)
  • Fix some crashes
@github-actions
github-actionsBot
commented
Sep 10, 2020

Copy link


Copy Markdown

https://issues.apache.org/jira/browse/ARROW-9465
@arw2019arw2019
mentioned this pull request
Sep 13, 2020
 Closed

jorisvandenbossche
jorisvandenbossche
reviewed
Sep 14, 2020
View reviewed changes
@jorisvandenbosschejorisvandenbossche
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvements!
Comment threadpython/pyarrow/_compute.pyx
Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should "meta" be explained here as well?
@pitroupitrouSep 14, 2020

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you're right.
Comment threadcpp/src/arrow/compute/api_aggregate.h
Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we simply change this? (is it public on the C++ side?) And it is changed for consistency with how it's called elsewhere?
@pitroupitrouSep 14, 2020

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The next version will be 2.0.0, meaning it's ok to break APIs. Of course, we should try to minimize breakage, but this one aims to fix an inconsistency.
@pitrou
ARROW-9465: [Python] Improve ergonomics of compute module
57ccea0
Lots of assorted things here:
* Automatically generate global wrappers for calling
registered compute functions
* Improve metadata of such wrappers (e.g. name, docstring)
* Make it easier to pass options (for example via kwargs)
* Type-check options
* Add some docstrings
* Expose more function attributes (e.g. arity)
* Fix some crashes
@pitrou
Add automatic scalar conversion
6e82d7b
@pitrou
Address review comment
648ee85
@pitrou
pitrouforce-pushed
the
ARROW-9465-py-compute-cosmetics
branch
from
7cce626 to
648ee85CompareSeptember 14, 2020 13:35
@pitrou
pitrou
commented
Sep 14, 2020

Copy link


Copy Markdown

MemberAuthor
CI failures here look unrelated (though someone should really take a look at them).
@pitrou
pitrou
deleted the

ARROW-9465-py-compute-cosmetics

branch
September 14, 2020 17:22
@maartenbreddelsmaartenbreddels
mentioned this pull request
Oct 6, 2020
 Closed

pitrou added a commit
that referenced
this pull request
Oct 8, 2020
@arw2019@pitrou
ARROW-9967: [Python] Add compute module docs + expose more option cla…
ba7ee65
…sses
#8163 exposes `pyarrow.compute` kernels and generates their docstrings. This PR adds documentation for the module in the User Guide and the Python API reference.
Closes#8145 from arw2019/ARROW-7871
Lead-authored-by: arw2019 <andrew.r.wieteska@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@asfimportasfimport
mentioned this pull request
Sep 14, 2020

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@pitrou@jorisvandenbossche

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
ARROW-9465: [Python] Improve ergonomics of compute module - #8163
Closed
pitrou wants to merge 4 commits into
apache:masterfrom 
pitrou:ARROW-9465-py-compute-cosmetics
Closed
ARROW-9465: [Python] Improve ergonomics of compute module#8163
pitrou wants to merge 4 commits into
apache:masterfrom 
pitrou:ARROW-9465-py-compute-cosmetics
Conversation
@pitrou
@pitroupitrou
commented
Sep 10, 2020

Copy link


Copy Markdown

Member
Lots of assorted things here:
  • Automatically generate global wrappers for calling
    
registered compute functions
  • Improve metadata of such wrappers (e.g. name, docstring)
  • Make it easier to pass options (for example via kwargs)
  • Type-check options
  • Add some docstrings
  • Expose more function attributes (e.g. arity)
  • Fix some crashes
@github-actions
github-actionsBot
commented
Sep 10, 2020

Copy link


Copy Markdown

https://issues.apache.org/jira/browse/ARROW-9465
@arw2019arw2019
mentioned this pull request
Sep 13, 2020
 Closed

jorisvandenbossche
jorisvandenbossche
reviewed
Sep 14, 2020
View reviewed changes
@jorisvandenbosschejorisvandenbossche
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvements!
Comment threadpython/pyarrow/_compute.pyx
Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should "meta" be explained here as well?
@pitroupitrouSep 14, 2020

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you're right.
Comment threadcpp/src/arrow/compute/api_aggregate.h
Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we simply change this? (is it public on the C++ side?) And it is changed for consistency with how it's called elsewhere?
@pitroupitrouSep 14, 2020

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The next version will be 2.0.0, meaning it's ok to break APIs. Of course, we should try to minimize breakage, but this one aims to fix an inconsistency.
@pitrou
ARROW-9465: [Python] Improve ergonomics of compute module
57ccea0
Lots of assorted things here:
* Automatically generate global wrappers for calling
registered compute functions
* Improve metadata of such wrappers (e.g. name, docstring)
* Make it easier to pass options (for example via kwargs)
* Type-check options
* Add some docstrings
* Expose more function attributes (e.g. arity)
* Fix some crashes
@pitrou
Add automatic scalar conversion
6e82d7b
@pitrou
Address review comment
648ee85
@pitrou
pitrouforce-pushed
the
ARROW-9465-py-compute-cosmetics
branch
from
7cce626 to
648ee85CompareSeptember 14, 2020 13:35
@pitrou
pitrou
commented
Sep 14, 2020

Copy link


Copy Markdown

MemberAuthor
CI failures here look unrelated (though someone should really take a look at them).
@pitrou
pitrou
deleted the

ARROW-9465-py-compute-cosmetics

branch
September 14, 2020 17:22
@maartenbreddelsmaartenbreddels
mentioned this pull request
Oct 6, 2020
 Closed

pitrou added a commit
that referenced
this pull request
Oct 8, 2020
@arw2019@pitrou
ARROW-9967: [Python] Add compute module docs + expose more option cla…
ba7ee65
…sses
#8163 exposes `pyarrow.compute` kernels and generates their docstrings. This PR adds documentation for the module in the User Guide and the Python API reference.
Closes#8145 from arw2019/ARROW-7871
Lead-authored-by: arw2019 <andrew.r.wieteska@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@asfimportasfimport
mentioned this pull request
Sep 14, 2020

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@pitrou@jorisvandenbossche

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
ARROW-9465: [Python] Improve ergonomics of compute module - #8163
Closed
pitrou wants to merge 4 commits into
apache:masterfrom 
pitrou:ARROW-9465-py-compute-cosmetics
Closed
ARROW-9465: [Python] Improve ergonomics of compute module#8163
pitrou wants to merge 4 commits into
apache:masterfrom 
pitrou:ARROW-9465-py-compute-cosmetics
Conversation
@pitrou
@pitroupitrou
commented
Sep 10, 2020

Copy link


Copy Markdown

Member
Lots of assorted things here:
  • Automatically generate global wrappers for calling
    
registered compute functions
  • Improve metadata of such wrappers (e.g. name, docstring)
  • Make it easier to pass options (for example via kwargs)
  • Type-check options
  • Add some docstrings
  • Expose more function attributes (e.g. arity)
  • Fix some crashes
@github-actions
github-actionsBot
commented
Sep 10, 2020

Copy link


Copy Markdown

https://issues.apache.org/jira/browse/ARROW-9465
@arw2019arw2019
mentioned this pull request
Sep 13, 2020
 Closed

jorisvandenbossche
jorisvandenbossche
reviewed
Sep 14, 2020
View reviewed changes
@jorisvandenbosschejorisvandenbossche
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvements!
Comment threadpython/pyarrow/_compute.pyx
Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should "meta" be explained here as well?
@pitroupitrouSep 14, 2020

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you're right.
Comment threadcpp/src/arrow/compute/api_aggregate.h
Outdated

@jorisvandenbosschejorisvandenbosscheSep 14, 2020

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we simply change this? (is it public on the C++ side?) And it is changed for consistency with how it's called elsewhere?
@pitroupitrouSep 14, 2020

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The next version will be 2.0.0, meaning it's ok to break APIs. Of course, we should try to minimize breakage, but this one aims to fix an inconsistency.
@pitrou
ARROW-9465: [Python] Improve ergonomics of compute module
57ccea0
Lots of assorted things here:
* Automatically generate global wrappers for calling
registered compute functions
* Improve metadata of such wrappers (e.g. name, docstring)
* Make it easier to pass options (for example via kwargs)
* Type-check options
* Add some docstrings
* Expose more function attributes (e.g. arity)
* Fix some crashes
@pitrou
Add automatic scalar conversion
6e82d7b
@pitrou
Address review comment
648ee85
@pitrou
pitrouforce-pushed
the
ARROW-9465-py-compute-cosmetics
branch
from
7cce626 to
648ee85CompareSeptember 14, 2020 13:35
@pitrou
pitrou
commented
Sep 14, 2020

Copy link


Copy Markdown

MemberAuthor
CI failures here look unrelated (though someone should really take a look at them).
@pitrou
pitrou
deleted the

ARROW-9465-py-compute-cosmetics

branch
September 14, 2020 17:22
@maartenbreddelsmaartenbreddels
mentioned this pull request
Oct 6, 2020
 Closed

pitrou added a commit
that referenced
this pull request
Oct 8, 2020
@arw2019@pitrou
ARROW-9967: [Python] Add compute module docs + expose more option cla…
ba7ee65
…sses
#8163 exposes `pyarrow.compute` kernels and generates their docstrings. This PR adds documentation for the module in the User Guide and the Python API reference.
Closes#8145 from arw2019/ARROW-7871
Lead-authored-by: arw2019 <andrew.r.wieteska@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@asfimportasfimport
mentioned this pull request
Sep 14, 2020

Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@jorisvandenbosschejorisvandenbosschejorisvandenbossche left review comments

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@pitrou@jorisvandenbossche