Skip to content

ARROW-13772: [R] Binding for median aggregation - #11018

Closed
thisisnic wants to merge 19 commits into
apache:masterfrom
thisisnic:ARROW-13772_median_hash
Closed

ARROW-13772: [R] Binding for median aggregation#11018
thisisnic wants to merge 19 commits into
apache:masterfrom
thisisnic:ARROW-13772_median_hash

Conversation

@thisisnic

@thisisnicthisisnic commented Aug 27, 2021•
edited by ianmcook
Loading

Copy link
Copy Markdown
Member

This adds a binding for median() in dplyr::summarise(). This also adds a binding for quantile() but only for a single probability per function call (length(probs) == 1). With both bindings, the results are approximate, calculated using the tdigest algorithm. The user is warned once per session that the results are approximate.

@github-actions

github-actionsBot commented Aug 27, 2021

Copy link
Copy Markdown

https://issues.apache.org/jira/browse/ARROW-13772

@thisisnic
thisisnic marked this pull request as draft August 27, 2021 11:02
@thisisnic

thisisnic commented Aug 27, 2021

Copy link
Copy Markdown
MemberAuthor

hash_digest returns a list, and currently we can't combine expression on aggregation kernels - look at this again after #10992 has been merged as this will enable this behaviour

@ianmcook
ianmcookforce-pushed the ARROW-13772_median_hash branch from ee8c6a2 to c4db4f1CompareSeptember 17, 2021 00:52
@ianmcook

ianmcook commented Sep 17, 2021

Copy link
Copy Markdown
Member

This is blocked until #11159 merges.

ianmcook
ianmcook reviewed Sep 17, 2021
View reviewed changes
Comment threadr/R/dplyr-summarize.R Outdated
@ianmcook
ianmcookforce-pushed the ARROW-13772_median_hash branch from 9515eeb to c44d54bCompareSeptember 20, 2021 20:48
@ianmcook
ianmcook marked this pull request as ready for review September 21, 2021 02:20
ianmcook
ianmcook reviewed Sep 21, 2021
View reviewed changes
Comment threadr/tests/testthat/test-dplyr-summarize.R Outdated
nealrichardson
nealrichardson reviewed Sep 21, 2021
View reviewed changes
Comment threadr/R/dplyr-summarize.R Outdated
Comment threadr/tests/testthat/test-dplyr-summarize.R Outdated
Comment threadr/R/dplyr-functions.R Outdated
Comment threadr/R/dplyr-functions.R Outdated
Comment threadr/R/dplyr-summarize.R Outdated
Comment threadr/R/dplyr-summarize.R Outdated
@ianmcook
ianmcook marked this pull request as draft September 21, 2021 22:04
@ianmcookianmcook mentioned this pull request Sep 21, 2021
Closed
@ianmcook

ianmcook commented Sep 21, 2021•
edited
Loading

Copy link
Copy Markdown
Member

I will rebase this after #11204 merges. Until then, tests will fail.

@ianmcook
ianmcookforce-pushed the ARROW-13772_median_hash branch from 421abf6 to a837785CompareSeptember 24, 2021 15:01
@ianmcook

ianmcook commented Sep 24, 2021

Copy link
Copy Markdown
Member

I addressed all comments and rebased after the approximate_median kernel PR was merged. I will merge when the CI is all green.

@ianmcook
ianmcook marked this pull request as ready for review September 24, 2021 15:41
@ianmcookianmcook mentioned this pull request Sep 25, 2021
Closed
ianmcook added a commit that referenced this pull request Sep 25, 2021
@ianmcook
MINOR: [R] Fix quantile() test failure in test-r-versions CI task
adb0190
This is a trivially small fix to resolve a test failure on older versions of R following ARROW-13772 (#11018).
Closes#11235 from ianmcook/ARROW-13772-fix
Authored-by: Ian Cook <ianmcook@gmail.com>
Signed-off-by: Ian Cook <ianmcook@gmail.com>
@ianmcookianmcook mentioned this pull request Oct 3, 2021
Closed
ViniciusSouzaRoque pushed a commit to s1mbi0se/arrow that referenced this pull request Oct 20, 2021
@ianmcook@thisisnic@ViniciusSouzaRoque
ARROW-13772: [R] Binding for median aggregation
548487b
This adds a binding for `median()` in `dplyr::summarise()`. This also adds a binding for `quantile()` but only for a single probability per function call (`length(probs) == 1`). With both bindings, the results are approximate, calculated using the tdigest algorithm. The user is warned once per session that the results are approximate.
Closesapache#11018 from thisisnic/ARROW-13772_median_hash
Lead-authored-by: Ian Cook <ianmcook@gmail.com>
Co-authored-by: Nic Crane <thisisnic@gmail.com>
Signed-off-by: Ian Cook <ianmcook@gmail.com>
ViniciusSouzaRoque pushed a commit to s1mbi0se/arrow that referenced this pull request Oct 20, 2021
@ianmcook@ViniciusSouzaRoque
MINOR: [R] Fix quantile() test failure in test-r-versions CI task
3d96177
This is a trivially small fix to resolve a test failure on older versions of R following ARROW-13772 (apache#11018).
Closesapache#11235 from ianmcook/ARROW-13772-fix
Authored-by: Ian Cook <ianmcook@gmail.com>
Signed-off-by: Ian Cook <ianmcook@gmail.com>
@asfimportasfimport mentioned this pull request Sep 25, 2021
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ianmcookianmcookianmcook left review comments
@nealrichardsonnealrichardsonnealrichardson left review comments

Assignees

No one assigned

Labels

Component: R

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@thisisnic@ianmcook@nealrichardson