Skip to content

Remove ScalarFunctionDefinition - #10325

Merged
alamb merged 6 commits into
apache:mainfrom
lewiszlw:remove-ScalarFunctionDefinition
May 7, 2024
Merged

Remove ScalarFunctionDefinition#10325
alamb merged 6 commits into
apache:mainfrom
lewiszlw:remove-ScalarFunctionDefinition

Conversation

@lewiszlw

Copy link
Copy Markdown
Member

Which issue does this PR close?

Closes#10175.

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actionsgithub-actionsBot added logical-expr Logical plan and expressions physical-expr Changes to the physical-expr crates optimizer Optimizer rules core Core DataFusion crate labels May 1, 2024
Comment threaddatafusion/expr/src/expr_schema.rs Outdated

@jayzhan211jayzhan211 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@Weijun-HWeijun-H left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks @lewiszlw

@alamb

alamb commented May 2, 2024

Copy link
Copy Markdown
Contributor

🚀

@alambalamb added the api change Changes the API exposed to users of the crate label May 2, 2024

@alambalamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @lewiszlw

Comment threaddatafusion/expr/src/expr.rs Outdated
@alamb

alamb commented May 2, 2024

Copy link
Copy Markdown
Contributor

It might be nice to wait to merge this PR until we make the #10217 release, to minimize API churn as 38.0.0 already removes BuiltInScalarFunction

@github-actionsgithub-actionsBot added the sql SQL Planner label May 2, 2024
@lewiszlw

Copy link
Copy Markdown
MemberAuthor

Clippy failed on other code. I run clippy locally and more than one warning needed to be addressed, might be done in another pr.

@alamb

alamb commented May 2, 2024

Copy link
Copy Markdown
Contributor

Clippy failed on other code. I run clippy locally and more than one warning needed to be addressed, might be done in another pr.

I think it may be due to the new rust https://releases.rs/docs/1.78.0/

will file a ticket

update: filed #10352

@alamb

alamb commented May 2, 2024

Copy link
Copy Markdown
Contributor

I merged up from main to pick up the fix for #10352

@lewiszlw
lewiszlw requested a review from alambMay 7, 2024 16:25
@alamb

alamb commented May 7, 2024

Copy link
Copy Markdown
Contributor

Now that #10407 is merged, I believe main is open for 38.0.0 business !

🚀 -- tanks again @lewiszlw

@alamb
alamb merged commit 742e3c5 into apache:mainMay 7, 2024
Michael-J-Ward added a commit to Michael-J-Ward/datafusion-python that referenced this pull request Jun 13, 2024
The field `func_name` was changed to `func` as part of removing `ScalarFunctionDefinition` upstream.
Ref apache/datafusion#10325
andygrove pushed a commit to apache/datafusion-python that referenced this pull request Jun 14, 2024
* deps: update datafusion to 39.0.0, pyo3 to 0.21, and object_store to 0.10.1
`datafusion-common` also depends on `pyo3`, so they need to be upgraded together.
* feat: remove GetIndexField
datafusion replaced Expr::GetIndexField with a FieldAccessor trait.
Ref apache/datafusion#10568
Ref apache/datafusion#10769
* feat: update ScalarFunction
The field `func_name` was changed to `func` as part of removing `ScalarFunctionDefinition` upstream.
Ref apache/datafusion#10325
* feat: incorporate upstream array_slice fixes
Fixes#670
* update ExectionPlan::children impl for DatasetExec
Ref apache/datafusion#10543
* update value_interval_daytime
Ref apache/arrow-rs#5769
* update regexp_replace and regexp_match
Fixes#677
* add gil-refs feature to pyo3
This silences pyo3's deprecation warnings for its new Bounds api.
It's the 1st step of the migration, and should be removed before merge.
Ref https://pyo3.rs/v0.21.0/migration#from-020-to-021
* fix signature for octet_length
Ref apache/datafusion#10726
* update signature for covar_samp
AggregateUDF expressions now have a builder API design, which removes arguments like filter and order_by
Ref apache/datafusion#10545
Ref apache/datafusion#10492
* convert covar_pop to expr_fn api
Ref: https://github.com/apache/datafusion/pull/10418/files
* convert median to expr_fn api
Ref apache/datafusion#10644
* convert variance sample to UDF
Ref apache/datafusion#10667
* convert first_value and last_value to UDFs
Ref apache/datafusion#10648
* checkpointing with a few todos to fix remaining compile errors
* impl PyExpr::python_value for IntervalDayTime and IntervalMonthDayNano
* convert sum aggregate function to UDF
* remove unnecessary clone on double reference
* apply cargo fmt
* remove duplicate allow-dead-code annotation
* update tpch examples for new pyarrow interval
Fixes#665
* marked q11 tpch example as expected fail
Ref #730
* add default stride of None back to array_slice
findepi pushed a commit to findepi/datafusion that referenced this pull request Jul 16, 2024
* Remove ScalarFunctionDefinition
* Fix test
* rename func_def to func
---------
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api changeChanges the API exposed to users of the cratecoreCore DataFusion cratelogical-exprLogical plan and expressionsoptimizerOptimizer rulesphysical-exprChanges to the physical-expr cratessqlSQL Planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove ScalarFunctionDefinition

4 participants

@lewiszlw@alamb@jayzhan211@Weijun-H