Skip to content

Introduce array_except function - #8135

Merged
alamb merged 5 commits into
apache:mainfrom
jayzhan211:array-except
Nov 17, 2023
Merged

Introduce array_except function#8135
alamb merged 5 commits into
apache:mainfrom
jayzhan211:array-except

Conversation

@jayzhan211

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes#6979

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actionsgithub-actionsBot added logical-expr Logical plan and expressions physical-expr Changes to the physical-expr crates sqllogictest SQL Logic Tests (.slt) labels Nov 11, 2023
parse_expr(&args[0], registry)?,
parse_expr(&args[1], registry)?,
)),
ScalarFunction::ArrayIntersect => Ok(array_intersect(

@jayzhan211jayzhan211Nov 11, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing from #8081. Not sure why CI from #8081 does not catch this

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to enhance the check during CI? maybe this is a little bug?


## array_except

statement ok

@jayzhan211jayzhan211Nov 11, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unlike other tests, I try a new style of test, keep creating table and drop table close together, hope this is easier for review.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is choice of style, imho no-table tests are easier to read.

query ?
select array_expect([], []), array_except(....)
----
[], ...

@jayzhan211jayzhan211Nov 13, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, we still need table to test multi-rows cases. The new style I mean is not create table in the beginning of this large file and drop table at the end of the file, but when we are done we drop it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the multirow still possible to do without creating a table

select ... from (select 1 a, 'asdf' b union all select 2 b, 'zxcv' b)

thats the matter of style of course.

@jayzhan211jayzhan211Nov 14, 2023

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We cant differentiate empty array and null currently. Both of them are null type So we can fix them in another PR

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also prefer the new style (create/query/drop table statements are organized more tightly). This will make tests more clear.

@jayzhan211
jayzhan211 marked this pull request as ready for review November 11, 2023 06:05

@compheadcomphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jayzhan211 please also add tests with empty arrays

And since we have introduced the new built in function we need to get it documented in
scalar_functions.md and expressions.md

@jayzhan211
jayzhan211 marked this pull request as draft November 13, 2023 23:33
@jayzhan211
jayzhan211 marked this pull request as ready for review November 14, 2023 01:31
Comment threaddocs/source/user-guide/sql/scalar_functions.md Outdated
Comment threaddocs/source/user-guide/sql/scalar_functions.md Outdated
@Veeupup

Copy link
Copy Markdown
Contributor

LGTM!

@compheadcomphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm thanks @jayzhan211
Please create a followup ticket to differentiate empty array and null, that sounds important

@alamb

Copy link
Copy Markdown
Contributor

There appears to be a non trivial number of conflicts in this PR now

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
jayzhan211and others added 2 commits November 15, 2023 09:05
@alamb

Copy link
Copy Markdown
Contributor

I took the liberty of merging up from main to resolve a merge conflict

@alamb
alamb merged commit 7293764 into apache:mainNov 17, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logical-exprLogical plan and expressionsphysical-exprChanges to the physical-expr cratessqllogictestSQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement array_except function

4 participants

@jayzhan211@Veeupup@alamb@comphead