Uh oh!
There was an error while loading. Please reload this page.
feat: impl the basic string_agg function - #8148
Conversation
| a | ||
| query TTTT | ||
| SELECT STRING_AGG('a',','), STRING_AGG('a', NULL), STRING_AGG(NULL, ','), STRING_AGG(NULL, NULL) |
There was a problem hiding this comment.
In postgresql, select string_agg('a', NULL) return a
but in duckdb, select string_agg('a', NULL) return NULL
I implemented it according to postgresql.
| NULL NULL 1 NULL 5 15 0 0 0 | ||
| 3 0 2 1 5.5 16.5 0.5 4.5 1.5 | ||
| 3 0 3 1 6 18 2 18 6 | ||
There was a problem hiding this comment.
I ported these tests over from duckdb.
alamb
commented
Nov 14, 2023
cc @universalmind303 -- do you have time to help review this PR? |
merge main
2010YOUY01
commented
Nov 15, 2023
Thank you! The implementation looks good to me, one minor suggestion is to add some empty string cases to sqllogictest like Also adopting test cases from mature systems like Postgres/Duckdb is a really good idea 👍🏼 , we should encourage contributors to do so when adding new functions |
haohuaijin
commented
Nov 15, 2023
Thanks for this example @2010YOUY01 , I found a bug in handling empty strings and nulls, which has now been fixed and added additional tests. |
haohuaijin
commented
Nov 15, 2023
the ci break maybe related to #8169 |
ozankabak
commented
Nov 15, 2023
@haohuaijin, indeed seems like it. We will fix promptly |
haohuaijin
commented
Nov 15, 2023
@ozankabak I've submitted a PR to try to fix it #8186 |
ozankabak
commented
Nov 15, 2023
Thank you for the hotfix, for some reason we didn't realize the tweak needed to accommodate #8034 |
ozankabak
commented
Nov 15, 2023
I will approve the hotfix once CI passes |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Co-authored-by: universalmind303 <universalmind.candy@gmail.com>
Co-authored-by: universalmind303 <universalmind.candy@gmail.com>
haohuaijin
commented
Nov 17, 2023
PTAL @alamb |
alamb
left a comment
There was a problem hiding this comment.
Thanks @haohuaijin this looks good to me -- I have a few small suggestions, but overall I think it looks good.
One thing I thought of was is I think this function will be non deterministic when aggregating without an ORDER BY -- specifically since the aggregate can see the inputs in any order when doing a multi-phase repartitioned group by.
I don't think we need to fix this now, but we should probably file a ticket to fix it
Thanks again
Uh oh!
There was an error while loading. Please reload this page.
| impl Accumulator for StringAggAccumulator { | ||
| fn update_batch(&mut self, values: &[ArrayRef]) -> Result<()> { | ||
| let string_array: Vec<_> = as_generic_string_array::<i64>(&values[0])? |
There was a problem hiding this comment.
using i64 means this will always take LargeStringArray, right? Shouldn't this use i32 for StringArray and i64 for LargeStringArray? However, the tests seem to cover this so this one looks good to me
There was a problem hiding this comment.
I do some cast like in sum i32 -> i64, so when we input a StringArray, cast it to a LargeStringArray.
| SELECT 'text1'::varchar(1000) as my_column union all | ||
| SELECT 'text1'::varchar(1000) as my_column union all | ||
| SELECT 'text1'::varchar(1000) as my_column |
There was a problem hiding this comment.
| SELECT 'text1'::varchar(1000) as my_column union all | |
| SELECT 'text1'::varchar(1000) as my_column union all | |
| SELECT 'text1'::varchar(1000) as my_column | |
| SELECT 'text1'::varchar(1000) as my_column union all | |
| SELECT 'text2'::varchar(1000) as my_column union all | |
| SELECT 'text3'::varchar(1000) as my_column |
There was a problem hiding this comment.
with out order by inside string_agg, the result is non deterministic, so I keep text1.
Uh oh!
There was an error while loading. Please reload this page.
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
haohuaijin
commented
Nov 18, 2023
track in #8260 |
alamb
commented
Nov 18, 2023
Thanks @haohuaijin@universalmind303 and @2010YOUY01 |
Which issue does this PR close?
part of #7910
Rationale for this change
add a new aggregate function
string_aggWhat changes are included in this PR?
Implement the basic
string_aggfunction, which currently does not supportdistinctandorder byclauses.Support for
distinctandorder bywill be added in future pr.Are these changes tested?
Are there any user-facing changes?