Skip to content

Add corr aggregate function - #1561

Merged
alamb merged 5 commits into
apache:masterfrom
realno:add-corr-operator
Jan 16, 2022
Merged

Add corr aggregate function#1561
alamb merged 5 commits into
apache:masterfrom
realno:add-corr-operator

Conversation

@realno

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

This PR add the corr function which is part of #1486

Rationale for this change

Add correlation function to DataFusion.

What changes are included in this PR?

New function corr(col1, co2) is added.

Are there any user-facing changes?

New function corr(col1, co2) is added.

No breaking change.

@github-actionsgithub-actionsBot added the documentation Improvements or additions to documentation label Jan 14, 2022
@liukun4515

liukun4515 commented Jan 14, 2022

Copy link
Copy Markdown
Contributor

why not support covar_pop and covar_samp together?
reference https://www.postgresql.org/docs/14/functions-aggregate.html
@realno

@realno

Copy link
Copy Markdown
ContributorAuthor

why not support covar_pop and covar_samp together? reference https://www.postgresql.org/docs/14/functions-aggregate.html@realno

covar_pop and covar_samp were merged here #1551

@liukun4515liukun4515 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
you can rebase the master, after the Clippy fixed

if let ScalarValue::Float64(Some(c)) = covar {
if let ScalarValue::Float64(Some(s1)) = stddev1 {
if let ScalarValue::Float64(Some(s2)) = stddev2 {
return Ok(ScalarValue::Float64(Some(c / s1 / s2)));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can s1 or s2 be 0?

@realnorealnoJan 16, 2022

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They may be 0, pushed a fix. Thanks for finding this.

@alamb
alamb merged commit b743610 into apache:masterJan 16, 2022
@alamb

Copy link
Copy Markdown
Contributor

Thanks @realno !

@realno
realno deleted the add-corr-operator branch February 9, 2022 20:09
@alambalamb changed the title add correlation functionAdd corr aggregate functionFeb 10, 2022
@alambalamb added enhancement New feature or request and removed documentation Improvements or additions to documentation labels Feb 10, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementNew feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@realno@liukun4515@alamb@jimexist@houqp