Skip to content

Support named query parameters - #8384

Merged
alamb merged 29 commits into
apache:mainfrom
Asura7969:query_params
Dec 4, 2023
Merged

Support named query parameters#8384
alamb merged 29 commits into
apache:mainfrom
Asura7969:query_params

Conversation

@Asura7969

@Asura7969Asura7969 commented Dec 1, 2023

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes#8245.

Rationale for this change

it might be clearer to use named parameters

What changes are included in this PR?

  • change the with_param_values API
  • add ParamValues struct
  • add test case

Are these changes tested?

test_named_query_parameters

Are there any user-facing changes?

Add named query,like:

let results = ctx
.sql("SELECT c1, c2 FROM test WHERE c1 > $coo AND c1 < $foo").await?
.with_param_values(vec![("foo",ScalarValue::UInt32(Some(3))),("coo",ScalarValue::UInt32(Some(0))),])?
.collect().await?;

@github-actionsgithub-actionsBot added sql SQL Planner logical-expr Logical plan and expressions core Core DataFusion crate labels Dec 1, 2023

/// The parameter value corresponding to the placeholder
#[derive(Debug, Clone)]
pub enum ParamValues {

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is more appropriate to encapsulate it into a struct

pub fn with_param_values(self, param_values: ParamValues) -> Result<LogicalPlan> {
match self {
LogicalPlan::Prepare(prepare_lp) => {
// Verify if the number of params matches the number of values

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to ParamValues(verify method)

expr.transform(&|expr| {
match &expr {
Expr::Placeholder(Placeholder { id, data_type }) => {
if id.is_empty() || id == "$0" {

@Asura7969Asura7969Dec 1, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to ParamValues(get_placeholders_with_values method)

@alamb

alamb commented Dec 1, 2023

Copy link
Copy Markdown
Contributor

Thanks @Asura7969 -- I hope to review this later today or tomorrow

@alambalamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Asura7969 -- this looks great 🙏 ❤️ -- it is embarassing that it wasn't immediately clear that this API can already handle named parameters.

I had a few comment suggestions to make it even more obvious, but I also think this PR could be merged as is and those comments done as follow ons.

Thanks again for the contribution


impl From<HashMap<String, ScalarValue>> for ParamValues {
fn from(value: HashMap<String, ScalarValue>) -> Self {
impl<K> From<Vec<(K, ScalarValue)>> for ParamValues

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌 very nice

/// ])?
/// .collect()
/// .await?;
/// assert_batches_eq!(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please also add a demonstration of how to use the hashmap version too so that it is easier to discover by a casual user?

Perhaps add this to the example:

/// // Note you can also provide named parameters/// let results = ctx/// .sql("SELECT a FROM example WHERE b = $my_param")/// .await?/// // replace $my_param with value 2/// // Note you can also use a HashMap as well/// .with_param_values(vec![/// "my_param",/// ScalarValue::from(2i64)/// ])?/// .collect()/// .await?;/// assert_batches_eq!(

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your comment, I'm done

Comment threaddatafusion/expr/src/logical_plan/plan.rs
@alamb

alamb commented Dec 4, 2023

Copy link
Copy Markdown
Contributor

Thanks @Asura7969 !

@alamb
alamb merged commit 37bbd66 into apache:mainDec 4, 2023

@alambalamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again @Asura7969 -- I also proposed a small follow on PR #8418 to add an example to LogicalPlan::with_parameters as well

@Asura7969
Asura7969 deleted the query_params branch December 5, 2023 01:03
appletreeisyellow pushed a commit to appletreeisyellow/datafusion that referenced this pull request Dec 15, 2023
* Minor: Improve the document format of JoinHashMap
* support named query parameters
* cargo fmt
* add `ParamValues` conversion
* improve doc
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

coreCore DataFusion cratelogical-exprLogical plan and expressionssqlSQL Planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support named query parameters

2 participants

@Asura7969@alamb