Skip to content

Unparse struct to sql - #13493

Merged
alamb merged 8 commits into
apache:mainfrom
velaskuez:unparse-struct-to-sql
Nov 21, 2024
Merged

Unparse struct to sql#13493
alamb merged 8 commits into
apache:mainfrom
velaskuez:unparse-struct-to-sql

Conversation

@velaskuez

@velaskuezvelaskuez commented Nov 19, 2024

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes#13477

Rationale for this change

Support roundtrip for statements containing named_struct and get_field

What changes are included in this PR?

Implement expr_to_sql for named_struct and get_field

Are these changes tested?

Yes

Are there any user-facing changes?

No

@github-actionsgithub-actionsBot added the sql SQL Planner label Nov 19, 2024
@velaskuez
velaskuez marked this pull request as draft November 19, 2024 19:34
@velaskuez
velaskuez marked this pull request as ready for review November 20, 2024 08:46

@jonathanc-njonathanc-n left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This look good to me! Just left some minor error changes.

Comment threaddatafusion/sql/src/unparser/expr.rs Outdated
ast::Expr::CompoundIdentifier(idents) => idents,
_ => unreachable!(),
},
_ => return internal_err!("get_field column is invalid"),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
_ => returninternal_err!("get_field column is invalid"),
_ => returninternal_err!("get_field expects first argument to be column, but received {:?}", args[0]),

Error message should be more clear here for feedback

Comment threaddatafusion/sql/src/unparser/expr.rs Outdated
Expr::Column(col) => match self.col_to_sql(col)? {
ast::Expr::Identifier(ident) => vec![ident],
ast::Expr::CompoundIdentifier(idents) => idents,
_ => unreachable!(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
_ => unreachable!(),
_ => returninternal_err!("get_field expects an Identifier or CompoundIdentifier, but received: {:?}", other)

I feel like unreachable should be avoided

@velaskuez

Copy link
Copy Markdown
ContributorAuthor

Thanks @jonathanc-n, I've updated the errors

@alambalamb added the unparser Changes to the unparser crate label Nov 20, 2024

@goldmedalgoldmedal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @delamarch3 overall looks good to me 👍 Only some suggestions.

Comment threaddatafusion/sql/src/unparser/expr.rs Outdated
Comment on lines +547 to +554
let mut id = match &args[0] {
Expr::Column(col) => match self.col_to_sql(col)? {
ast::Expr::Identifier(ident) => vec![ident],
ast::Expr::CompoundIdentifier(idents) => idents,
other => return internal_err!("expected col_to_sql to return an Identifier or CompoundIdentifier, but received: {:?}", other),
},
_ => return internal_err!("get_field expects first argument to be column, but received: {:?}", &args[0]),
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should consider the more complex cases, a struct contains a struct field. Consider the following SQL that uses another way to access the struct:

> explain SELECT s.a['a1'] FROM (SELECT {a:{a1:c1}, b:{a1:c1}} AS s from t1);
+---------------+-------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type | plan |
+---------------+-------------------------------------------------------------------------------------------------------------------------------------------+
| logical_plan | Projection: get_field(get_field(named_struct(Utf8("a"), __common_expr_1, Utf8("b"), __common_expr_1), Utf8("a")), Utf8("a1")) AS s[a][a1] |
| | Projection: named_struct(Utf8("a1"), t1.c1) AS __common_expr_1 |
| | TableScan: t1 projection=[c1] 

You can see the get_field accepts another get_field function. However, I think we can't do a roundtrip test for this kind of case currently because SQL like select s.a.a1 isn't supported (apache/datafusion-sqlparser-rs#1541 will address it).

By the way, I think we can file a follow-up issue for the nested case. We don't need to support it in this PR.

Co-authored-by: Jax Liu <liugs963@gmail.com>
@velaskuez

Copy link
Copy Markdown
ContributorAuthor

Thanks for the review @goldmedal

@alamb
alamb merged commit e7d9504 into apache:mainNov 21, 2024
@alamb

Copy link
Copy Markdown
Contributor

🚀

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sqlSQL PlannerunparserChanges to the unparser crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unparse Struct plan to SQL string

4 participants

@velaskuez@alamb@goldmedal@jonathanc-n