Skip to content

[fix](nereids)support group_concat with distinct and order by - #38080

Merged
starocean999 merged 4 commits into
apache:masterfrom
starocean999:nereids_1222
Aug 5, 2024
Merged

[fix](nereids)support group_concat with distinct and order by#38080
starocean999 merged 4 commits into
apache:masterfrom
starocean999:nereids_1222

Conversation

@starocean999

@starocean999starocean999 commented Jul 18, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Proposed changes

This pr support group_concat with distinct and order by. So we can support sql like bellow:

select group_concat(distinct x order by y) from t;
select multi_distinct_group_concat(x order by y) from t;

@doris-robot

doris-robot commented Jul 18, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@starocean999

starocean999 commented Jul 30, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Jul 30, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 41914 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d2f9b651ee11d4d65a22bbd4860584b517878cc6, data reload: false
------ Round 1 ----------------------------------
q1	18323	4237	4166	4166
q2	2024	206	199	199
q3	10489	1266	1313	1266
q4	10168	851	955	851
q5	7614	3076	3000	3000
q6	220	137	141	137
q7	1057	623	625	623
q8	9451	1961	1982	1961
q9	10194	6682	6682	6682
q10	8796	3860	3851	3851
q11	430	256	254	254
q12	437	234	232	232
q13	17779	2923	2959	2923
q14	278	244	235	235
q15	523	493	492	492
q16	525	390	387	387
q17	988	909	881	881
q18	8015	7308	7219	7219
q19	1389	1223	1232	1223
q20	587	319	349	319
q21	5340	4734	4777	4734
q22	354	279	280	279
Total cold run time: 114981 ms
Total hot run time: 41914 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4052	4018	4051	4018
q2	330	234	222	222
q3	2991	3014	2995	2995
q4	1929	1898	1886	1886
q5	5261	5260	5246	5246
q6	218	134	136	134
q7	2098	1678	1693	1678
q8	3245	3318	3313	3313
q9	8343	8344	8313	8313
q10	3780	3840	3858	3840
q11	550	453	453	453
q12	726	586	567	567
q13	10021	2975	2939	2939
q14	279	254	253	253
q15	519	485	474	474
q16	449	388	385	385
q17	1736	1712	1708	1708
q18	7734	7275	7247	7247
q19	1671	1671	1675	1671
q20	1956	1741	1777	1741
q21	5471	5298	5247	5247
q22	538	456	462	456
Total cold run time: 63897 ms
Total hot run time: 54786 ms

@doris-robot

doris-robot commented Jul 30, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 168737 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d2f9b651ee11d4d65a22bbd4860584b517878cc6, data reload: false
query1	919	384	367	367
query2	6476	1723	1733	1723
query3	6663	213	223	213
query4	20086	17311	17221	17221
query5	4298	534	519	519
query6	279	191	181	181
query7	4597	302	302	302
query8	256	199	204	199
query9	8510	2407	2363	2363
query10	430	268	267	267
query11	10554	9993	10002	9993
query12	138	91	85	85
query13	1618	372	370	370
query14	8894	7046	7938	7046
query15	200	166	163	163
query16	7120	467	476	467
query17	929	560	533	533
query18	1907	286	280	280
query19	196	146	145	145
query20	90	86	87	86
query21	206	99	94	94
query22	4203	4077	3913	3913
query23	33766	33017	32945	32945
query24	10371	3080	3000	3000
query25	689	386	378	378
query26	1805	149	158	149
query27	3017	277	274	274
query28	7045	1960	1954	1954
query29	1369	422	413	413
query30	284	153	151	151
query31	920	756	764	756
query32	101	53	54	53
query33	700	309	309	309
query34	920	479	510	479
query35	845	748	754	748
query36	1009	886	858	858
query37	299	77	76	76
query38	2848	2750	2755	2750
query39	880	809	809	809
query40	282	110	113	110
query41	50	44	44	44
query42	120	100	111	100
query43	495	458	438	438
query44	1208	735	728	728
query45	211	175	172	172
query46	1089	822	791	791
query47	1788	1755	1707	1707
query48	374	295	291	291
query49	1189	437	417	417
query50	919	434	449	434
query51	6787	6748	6607	6607
query52	103	99	92	92
query53	262	178	178	178
query54	641	462	459	459
query55	76	74	76	74
query56	271	263	286	263
query57	1155	1052	1037	1037
query58	274	290	261	261
query59	2648	2417	2522	2417
query60	314	288	266	266
query61	97	95	96	95
query62	920	660	658	658
query63	220	182	183	182
query64	5907	1935	1882	1882
query65	3151	3113	3109	3109
query66	1456	349	334	334
query67	15253	15093	14760	14760
query68	4288	563	576	563
query69	453	302	295	295
query70	1096	1041	1067	1041
query71	414	276	275	275
query72	7147	2712	2529	2529
query73	759	330	325	325
query74	6022	5542	5527	5527
query75	3379	2734	2817	2734
query76	2254	1361	1430	1361
query77	451	321	309	309
query78	9398	8940	8893	8893
query79	1946	540	540	540
query80	1144	507	513	507
query81	551	223	225	223
query82	1199	135	127	127
query83	259	170	170	170
query84	268	79	80	79
query85	1381	327	308	308
query86	390	306	294	294
query87	3297	3085	3099	3085
query88	2936	2425	2411	2411
query89	380	294	303	294
query90	1807	197	192	192
query91	129	102	103	102
query92	63	49	50	49
query93	1617	623	619	619
query94	938	290	290	290
query95	380	278	267	267
query96	605	280	277	277
query97	3174	3113	3023	3023
query98	223	194	197	194
query99	1678	1316	1296	1296
Total cold run time: 262723 ms
Total hot run time: 168737 ms

@doris-robot

doris-robot commented Jul 30, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 29.81 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d2f9b651ee11d4d65a22bbd4860584b517878cc6, data reload: false
query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.08	0.09
query5	0.50	0.49	0.49
query6	1.14	0.71	0.71
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.57	0.51	0.51
query10	0.57	0.57	0.56
query11	0.16	0.11	0.12
query12	0.14	0.12	0.12
query13	0.61	0.60	0.59
query14	0.78	0.80	0.78
query15	0.90	0.86	0.85
query16	0.36	0.35	0.36
query17	0.96	1.00	0.97
query18	0.23	0.22	0.22
query19	1.84	1.70	1.73
query20	0.01	0.00	0.01
query21	15.44	0.73	0.65
query22	4.02	7.68	1.12
query23	17.95	1.34	1.30
query24	2.26	0.22	0.22
query25	0.19	0.08	0.07
query26	0.32	0.22	0.22
query27	0.45	0.24	0.23
query28	13.15	1.00	0.96
query29	12.54	3.33	3.34
query30	0.26	0.06	0.05
query31	2.87	0.41	0.41
query32	3.25	0.50	0.49
query33	2.87	2.96	2.92
query34	15.47	4.26	4.24
query35	4.30	4.28	4.28
query36	0.68	0.48	0.48
query37	0.18	0.16	0.16
query38	0.17	0.15	0.15
query39	0.04	0.04	0.04
query40	0.16	0.13	0.13
query41	0.11	0.05	0.05
query42	0.05	0.06	0.06
query43	0.05	0.04	0.05
Total cold run time: 107.65 s
Total hot run time: 29.81 s

@starocean999

starocean999 commented Jul 31, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Jul 31, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 41761 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9fa7a8c4e185cf0d79e9b179fce39e4fb84794af, data reload: false
------ Round 1 ----------------------------------
q1	17596	4119	4036	4036
q2	2018	204	206	204
q3	10462	1252	1329	1252
q4	10162	832	912	832
q5	7624	3034	2996	2996
q6	221	138	140	138
q7	1038	624	632	624
q8	9444	1926	1984	1926
q9	8582	6631	6625	6625
q10	8743	3828	3825	3825
q11	427	256	258	256
q12	410	233	227	227
q13	17772	2946	2952	2946
q14	274	247	246	246
q15	539	480	479	479
q16	526	395	386	386
q17	983	933	906	906
q18	7979	7290	7272	7272
q19	1473	1220	1212	1212
q20	566	336	327	327
q21	5323	4801	4768	4768
q22	356	286	278	278
Total cold run time: 112518 ms
Total hot run time: 41761 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4060	4004	4025	4004
q2	325	228	223	223
q3	2958	2986	3107	2986
q4	1997	2032	1999	1999
q5	5620	5539	5457	5457
q6	222	141	133	133
q7	2086	1727	1852	1727
q8	3340	3412	3344	3344
q9	8646	8694	8754	8694
q10	4010	4007	4037	4007
q11	548	449	453	449
q12	732	557	566	557
q13	13350	3094	3140	3094
q14	319	285	278	278
q15	534	495	505	495
q16	455	420	420	420
q17	1758	1757	1717	1717
q18	8194	7775	7767	7767
q19	1730	1722	1739	1722
q20	2060	1876	1880	1876
q21	5704	5494	5346	5346
q22	511	452	482	452
Total cold run time: 69159 ms
Total hot run time: 56747 ms

@starocean999

starocean999 commented Jul 31, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Jul 31, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 41436 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d7dcee50ed86c771a8710be561b7671ac57882a6, data reload: false
------ Round 1 ----------------------------------
q1	17601	4051	4016	4016
q2	2015	206	206	206
q3	10432	1281	1290	1281
q4	10169	814	918	814
q5	7637	2929	2953	2929
q6	220	140	142	140
q7	1021	607	617	607
q8	9433	1901	1903	1901
q9	8563	6595	6544	6544
q10	8737	3832	3829	3829
q11	443	259	261	259
q12	417	234	232	232
q13	17757	3600	2946	2946
q14	267	250	244	244
q15	519	479	485	479
q16	518	393	385	385
q17	941	910	927	910
q18	7922	7286	7180	7180
q19	3535	1212	1196	1196
q20	566	327	351	327
q21	5270	4737	4730	4730
q22	345	290	281	281
Total cold run time: 114328 ms
Total hot run time: 41436 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4115	3996	4026	3996
q2	328	227	220	220
q3	2981	3081	3120	3081
q4	2029	1994	1985	1985
q5	5577	5454	5427	5427
q6	219	130	133	130
q7	2134	1825	1796	1796
q8	3295	3333	3358	3333
q9	8723	8659	8828	8659
q10	3896	4050	3880	3880
q11	553	456	465	456
q12	782	591	618	591
q13	16268	3095	3107	3095
q14	298	293	287	287
q15	524	507	485	485
q16	458	413	433	413
q17	1772	1725	1732	1725
q18	8210	7716	7694	7694
q19	1713	1700	1701	1700
q20	2130	1827	1824	1824
q21	5753	5501	5240	5240
q22	515	464	494	464
Total cold run time: 72273 ms
Total hot run time: 56481 ms

@doris-robot

doris-robot commented Jul 31, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 170687 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d7dcee50ed86c771a8710be561b7671ac57882a6, data reload: false
query1	901	374	371	371
query2	6479	1760	1720	1720
query3	6673	211	224	211
query4	20692	17289	17472	17289
query5	3685	508	524	508
query6	272	172	168	168
query7	4586	308	295	295
query8	256	182	192	182
query9	8493	2372	2370	2370
query10	426	286	274	274
query11	10553	10033	9986	9986
query12	125	90	92	90
query13	1637	384	376	376
query14	9291	8030	8029	8029
query15	208	166	168	166
query16	7017	491	454	454
query17	975	606	564	564
query18	1915	298	292	292
query19	200	152	147	147
query20	94	87	89	87
query21	208	102	99	99
query22	4285	4081	4156	4081
query23	33715	34614	33419	33419
query24	10230	3108	3134	3108
query25	688	389	419	389
query26	1723	151	156	151
query27	3035	294	281	281
query28	7537	2015	2017	2015
query29	1260	441	444	441
query30	239	154	155	154
query31	978	826	740	740
query32	100	55	55	55
query33	688	332	326	326
query34	933	496	505	496
query35	859	780	776	776
query36	1043	867	901	867
query37	289	83	81	81
query38	2946	2818	2870	2818
query39	897	817	813	813
query40	267	117	116	116
query41	45	46	44	44
query42	125	100	99	99
query43	495	421	439	421
query44	1191	729	732	729
query45	228	174	176	174
query46	1120	805	798	798
query47	1758	1696	1702	1696
query48	365	297	284	284
query49	927	422	409	409
query50	911	439	435	435
query51	6827	6652	6654	6652
query52	102	90	87	87
query53	256	180	184	180
query54	625	450	463	450
query55	78	77	76	76
query56	278	264	259	259
query57	1130	1051	1030	1030
query58	267	262	276	262
query59	2724	2415	2434	2415
query60	298	272	271	271
query61	97	116	89	89
query62	892	664	644	644
query63	218	185	190	185
query64	5558	1907	1872	1872
query65	3174	3084	3109	3084
query66	1307	351	333	333
query67	15051	14704	14766	14704
query68	4281	574	575	574
query69	434	292	303	292
query70	1109	1067	1086	1067
query71	418	283	278	278
query72	7094	2737	2492	2492
query73	763	325	330	325
query74	5997	5620	5596	5596
query75	3407	2786	2741	2741
query76	2348	1370	1408	1370
query77	425	310	325	310
query78	9403	9027	8818	8818
query79	1864	546	560	546
query80	1158	526	508	508
query81	587	233	229	229
query82	1072	135	129	129
query83	238	177	169	169
query84	280	136	79	79
query85	1256	316	299	299
query86	393	310	321	310
query87	3273	3068	3112	3068
query88	3001	2424	2414	2414
query89	390	301	287	287
query90	1722	192	196	192
query91	124	102	100	100
query92	57	51	52	51
query93	1537	621	625	621
query94	811	300	295	295
query95	380	272	316	272
query96	595	289	278	278
query97	3247	3057	3026	3026
query98	218	199	195	195
query99	1634	1269	1297	1269
Total cold run time: 261961 ms
Total hot run time: 170687 ms

@doris-robot

doris-robot commented Jul 31, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 29.86 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d7dcee50ed86c771a8710be561b7671ac57882a6, data reload: false
query1	0.04	0.04	0.03
query2	0.08	0.03	0.04
query3	0.22	0.04	0.05
query4	1.69	0.07	0.06
query5	0.49	0.49	0.48
query6	1.14	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.58	0.52	0.50
query10	0.57	0.57	0.57
query11	0.15	0.12	0.11
query12	0.14	0.12	0.12
query13	0.61	0.60	0.60
query14	0.77	0.80	0.79
query15	0.90	0.87	0.85
query16	0.36	0.36	0.35
query17	0.99	1.65	0.98
query18	0.22	0.21	0.21
query19	1.84	1.71	1.76
query20	0.01	0.01	0.01
query21	15.40	0.77	0.64
query22	4.37	6.88	1.14
query23	18.49	1.42	1.39
query24	2.26	0.22	0.22
query25	0.19	0.07	0.08
query26	0.31	0.22	0.21
query27	0.46	0.24	0.22
query28	13.17	0.99	0.96
query29	12.53	3.32	3.32
query30	0.26	0.06	0.05
query31	2.88	0.41	0.41
query32	3.23	0.49	0.48
query33	2.85	2.95	2.98
query34	15.46	4.25	4.29
query35	4.27	4.29	4.30
query36	0.68	0.47	0.49
query37	0.19	0.16	0.17
query38	0.16	0.15	0.15
query39	0.04	0.04	0.03
query40	0.16	0.13	0.12
query41	0.09	0.04	0.05
query42	0.05	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 108.41 s
Total hot run time: 29.86 s

@starocean999

starocean999 commented Aug 2, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Aug 2, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 41780 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 60d2df934f65ee6be88a7a7a49543103957a1bd7, data reload: false
------ Round 1 ----------------------------------
q1	17676	4126	4118	4118
q2	2016	199	202	199
q3	10444	1337	1395	1337
q4	10160	852	960	852
q5	7646	2999	3014	2999
q6	223	141	144	141
q7	1066	619	612	612
q8	9431	1883	1940	1883
q9	8536	6617	6614	6614
q10	8743	3851	3849	3849
q11	433	251	243	243
q12	426	232	232	232
q13	17777	2921	2963	2921
q14	274	248	243	243
q15	518	484	480	480
q16	521	389	390	389
q17	982	929	924	924
q18	8093	7370	7244	7244
q19	1728	1191	1225	1191
q20	562	321	324	321
q21	5330	4705	4758	4705
q22	351	285	283	283
Total cold run time: 112936 ms
Total hot run time: 41780 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4131	4083	4047	4047
q2	333	227	220	220
q3	3022	3019	3194	3019
q4	1996	2015	1964	1964
q5	5633	5501	5432	5432
q6	221	132	132	132
q7	2119	1783	1810	1783
q8	3333	3369	3385	3369
q9	8672	8687	8866	8687
q10	3967	4087	3934	3934
q11	566	470	460	460
q12	766	614	610	610
q13	16693	3069	3093	3069
q14	308	282	270	270
q15	529	501	492	492
q16	459	407	407	407
q17	1785	1769	1713	1713
q18	8280	7866	7818	7818
q19	1751	1737	1730	1730
q20	2079	1854	1829	1829
q21	5772	5282	5595	5282
q22	531	445	478	445
Total cold run time: 72946 ms
Total hot run time: 56712 ms

@doris-robot

doris-robot commented Aug 2, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 168853 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 60d2df934f65ee6be88a7a7a49543103957a1bd7, data reload: false
query1	899	373	373	373
query2	6444	1753	1652	1652
query3	6659	209	220	209
query4	20158	17388	17548	17388
query5	3622	511	513	511
query6	269	169	157	157
query7	4604	296	296	296
query8	253	202	193	193
query9	8597	2366	2359	2359
query10	431	288	255	255
query11	10533	10179	10115	10115
query12	120	95	91	91
query13	1663	386	366	366
query14	9722	6539	6345	6345
query15	213	162	164	162
query16	6970	422	460	422
query17	936	577	535	535
query18	1907	285	289	285
query19	184	144	143	143
query20	88	85	81	81
query21	196	98	95	95
query22	4144	4001	3946	3946
query23	33883	33732	33371	33371
query24	10167	3096	3129	3096
query25	720	432	391	391
query26	1738	149	148	148
query27	3031	281	278	278
query28	7681	2010	2008	2008
query29	1305	422	425	422
query30	232	154	155	154
query31	969	778	751	751
query32	101	52	54	52
query33	683	314	324	314
query34	943	481	502	481
query35	883	770	774	770
query36	1038	860	877	860
query37	298	83	79	79
query38	2894	2907	2823	2823
query39	897	814	797	797
query40	251	110	112	110
query41	45	48	47	47
query42	129	108	95	95
query43	476	406	401	401
query44	1163	733	749	733
query45	217	180	178	178
query46	1111	792	769	769
query47	1872	1729	1721	1721
query48	365	293	290	290
query49	938	435	414	414
query50	914	432	432	432
query51	6824	6773	6617	6617
query52	95	95	91	91
query53	255	186	192	186
query54	627	455	455	455
query55	83	74	73	73
query56	265	253	251	251
query57	1150	1031	1052	1031
query58	259	274	285	274
query59	2652	2326	2232	2232
query60	300	274	271	271
query61	96	90	103	90
query62	894	659	651	651
query63	205	176	178	176
query64	5588	1889	1870	1870
query65	3140	3081	3076	3076
query66	1316	328	374	328
query67	15328	14888	14857	14857
query68	4339	563	591	563
query69	443	306	297	297
query70	1097	1071	1046	1046
query71	415	279	279	279
query72	7071	2659	2520	2520
query73	763	334	335	334
query74	6002	5704	5692	5692
query75	3409	2735	2753	2735
query76	2330	1184	1280	1184
query77	416	300	300	300
query78	9430	8952	8871	8871
query79	1345	523	523	523
query80	955	503	521	503
query81	553	223	233	223
query82	1051	130	130	130
query83	240	175	169	169
query84	264	79	81	79
query85	1231	307	320	307
query86	397	308	289	289
query87	3267	3126	3153	3126
query88	3014	2542	2527	2527
query89	381	303	290	290
query90	1701	202	201	201
query91	136	113	110	110
query92	59	52	50	50
query93	1372	623	608	608
query94	774	294	287	287
query95	403	283	274	274
query96	622	290	286	286
query97	3270	3079	3087	3079
query98	220	201	194	194
query99	1654	1373	1284	1284
Total cold run time: 261257 ms
Total hot run time: 168853 ms

@doris-robot

doris-robot commented Aug 2, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 29.62 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 60d2df934f65ee6be88a7a7a49543103957a1bd7, data reload: false
query1	0.05	0.04	0.04
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.69	0.07	0.07
query5	0.48	0.48	0.48
query6	1.13	0.71	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.51	0.51
query10	0.57	0.56	0.55
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.63	0.60	0.62
query14	0.78	0.80	0.79
query15	0.89	0.86	0.85
query16	0.35	0.36	0.35
query17	0.98	1.00	1.00
query18	0.22	0.21	0.21
query19	1.81	1.67	1.71
query20	0.01	0.02	0.01
query21	15.39	0.77	0.67
query22	4.34	8.17	0.99
query23	17.82	1.38	1.36
query24	2.23	0.21	0.22
query25	0.18	0.08	0.08
query26	0.32	0.21	0.21
query27	0.47	0.22	0.22
query28	13.22	1.00	0.96
query29	12.50	3.30	3.28
query30	0.26	0.06	0.05
query31	2.86	0.40	0.41
query32	3.25	0.50	0.48
query33	2.95	2.96	2.93
query34	15.44	4.24	4.24
query35	4.31	4.29	4.27
query36	0.67	0.49	0.48
query37	0.20	0.17	0.15
query38	0.16	0.15	0.14
query39	0.04	0.04	0.04
query40	0.16	0.13	0.13
query41	0.10	0.04	0.04
query42	0.05	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 107.79 s
Total hot run time: 29.62 s

@starocean999

starocean999 commented Aug 2, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Aug 2, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot

doris-robot commented Aug 2, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 41720 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ea90c5a6e5ad53759cc8e1392019e530a0153316, data reload: false
------ Round 1 ----------------------------------
q1	18303	4181	4222	4181
q2	2407	205	206	205
q3	11839	1363	1407	1363
q4	10540	852	943	852
q5	7871	2966	2987	2966
q6	226	140	138	138
q7	1075	618	622	618
q8	10028	1924	1937	1924
q9	8440	6579	6586	6579
q10	8717	3814	3847	3814
q11	433	257	265	257
q12	409	228	224	224
q13	17768	2959	2964	2959
q14	277	240	241	240
q15	536	473	495	473
q16	518	394	388	388
q17	951	926	902	902
q18	7970	7325	7156	7156
q19	1419	1223	1199	1199
q20	563	328	337	328
q21	5318	4674	4746	4674
q22	346	282	280	280
Total cold run time: 115954 ms
Total hot run time: 41720 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4041	4004	3980	3980
q2	326	229	226	226
q3	2979	3001	2968	2968
q4	1876	1847	1866	1847
q5	5232	5206	5237	5206
q6	217	130	129	129
q7	2061	1683	1685	1683
q8	3171	3267	3229	3229
q9	8264	8245	8196	8196
q10	3737	3830	3832	3830
q11	554	462	459	459
q12	713	559	538	538
q13	11052	2940	2930	2930
q14	300	261	253	253
q15	520	476	479	476
q16	442	402	396	396
q17	1727	1693	1683	1683
q18	7793	7425	7203	7203
q19	1669	1662	1666	1662
q20	2022	1755	1777	1755
q21	5395	5257	5263	5257
q22	525	479	463	463
Total cold run time: 64616 ms
Total hot run time: 54369 ms

@doris-robot

doris-robot commented Aug 2, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 168254 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ea90c5a6e5ad53759cc8e1392019e530a0153316, data reload: false
query1	920	384	375	375
query2	6502	1684	1673	1673
query3	6686	218	219	218
query4	19965	17057	17203	17057
query5	4297	529	505	505
query6	288	172	177	172
query7	4609	299	305	299
query8	256	192	195	192
query9	8516	2365	2343	2343
query10	439	271	268	268
query11	10619	10106	9921	9921
query12	138	94	87	87
query13	1637	386	396	386
query14	9309	6991	7549	6991
query15	204	160	166	160
query16	7130	443	453	443
query17	937	564	539	539
query18	1872	282	274	274
query19	186	140	138	138
query20	92	83	87	83
query21	201	102	100	100
query22	4392	4124	4189	4124
query23	33777	33057	32805	32805
query24	10333	3018	3021	3018
query25	672	384	396	384
query26	1735	150	151	150
query27	2889	277	276	276
query28	6858	1952	1938	1938
query29	1280	412	403	403
query30	288	150	152	150
query31	941	746	758	746
query32	98	55	57	55
query33	688	313	325	313
query34	903	477	485	477
query35	846	733	729	729
query36	984	848	872	848
query37	283	76	78	76
query38	2829	2770	2733	2733
query39	869	802	825	802
query40	282	111	114	111
query41	48	44	47	44
query42	120	126	104	104
query43	452	415	423	415
query44	1199	727	743	727
query45	209	179	177	177
query46	1075	809	780	780
query47	1828	1730	1703	1703
query48	369	306	291	291
query49	1190	422	405	405
query50	886	430	437	430
query51	6841	6768	6661	6661
query52	112	88	96	88
query53	263	190	177	177
query54	639	454	457	454
query55	75	72	74	72
query56	269	254	265	254
query57	1175	1020	1039	1020
query58	276	259	261	259
query59	2555	2278	2431	2278
query60	322	279	274	274
query61	95	93	94	93
query62	937	692	683	683
query63	214	183	183	183
query64	5897	1886	1878	1878
query65	3143	3083	3154	3083
query66	1431	337	330	330
query67	15312	14971	14984	14971
query68	4307	561	574	561
query69	448	308	300	300
query70	1111	1074	1041	1041
query71	409	273	279	273
query72	7126	2706	2490	2490
query73	760	331	333	331
query74	6053	5639	5652	5639
query75	3419	2763	2715	2715
query76	2447	1195	1265	1195
query77	429	299	312	299
query78	9365	8977	8849	8849
query79	1319	537	537	537
query80	1002	502	498	498
query81	543	230	227	227
query82	1052	127	133	127
query83	238	171	176	171
query84	280	79	78	78
query85	1288	431	291	291
query86	402	303	301	301
query87	3263	3089	3073	3073
query88	2990	2504	2500	2500
query89	410	298	290	290
query90	1788	188	188	188
query91	120	103	101	101
query92	60	51	49	49
query93	1415	617	616	616
query94	884	306	268	268
query95	372	257	265	257
query96	598	285	284	284
query97	3220	3072	3055	3055
query98	215	200	198	198
query99	1626	1290	1256	1256
Total cold run time: 261641 ms
Total hot run time: 168254 ms

@doris-robot

doris-robot commented Aug 2, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 29.97 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ea90c5a6e5ad53759cc8e1392019e530a0153316, data reload: false
query1	0.05	0.04	0.03
query2	0.07	0.03	0.04
query3	0.23	0.04	0.04
query4	1.66	0.07	0.07
query5	0.49	0.49	0.48
query6	1.14	0.72	0.71
query7	0.01	0.01	0.01
query8	0.05	0.04	0.04
query9	0.58	0.50	0.51
query10	0.58	0.57	0.57
query11	0.15	0.12	0.12
query12	0.15	0.13	0.12
query13	0.60	0.60	0.60
query14	0.76	0.79	0.79
query15	0.90	0.86	0.87
query16	0.35	0.34	0.35
query17	1.01	0.99	0.98
query18	0.23	0.21	0.21
query19	1.91	1.75	1.76
query20	0.02	0.01	0.01
query21	15.40	0.75	0.65
query22	4.02	9.07	1.32
query23	17.97	1.24	1.23
query24	2.26	0.22	0.23
query25	0.18	0.08	0.09
query26	0.31	0.20	0.21
query27	0.46	0.23	0.22
query28	13.19	1.00	0.97
query29	12.65	3.33	3.32
query30	0.26	0.08	0.06
query31	2.83	0.40	0.39
query32	3.26	0.49	0.48
query33	2.98	2.93	2.98
query34	15.45	4.28	4.27
query35	4.34	4.30	4.35
query36	0.69	0.48	0.47
query37	0.19	0.16	0.16
query38	0.17	0.16	0.15
query39	0.05	0.04	0.04
query40	0.16	0.14	0.13
query41	0.10	0.04	0.05
query42	0.06	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 107.97 s
Total hot run time: 29.97 s

@starocean999
starocean999 marked this pull request as ready for review August 5, 2024 02:21
@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Aug 5, 2024
@github-actions

github-actionsBot commented Aug 5, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Aug 5, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@starocean999starocean999 changed the title [fix](nereids)support group_concat with agg state[fix](nereids)support group_concat with distinct and order byAug 5, 2024
@starocean999
starocean999 merged commit f97de21 into apache:masterAug 5, 2024
@starocean999starocean999 mentioned this pull request Aug 5, 2024
Merged
yiguolei pushed a commit that referenced this pull request Aug 5, 2024
@starocean999
[fix](nereids)support group_concat with distinct and order by (#38871)
40567b5
## Proposed changes
pick from master #38080
<!--Describe your changes.-->
dataroaring pushed a commit that referenced this pull request Aug 11, 2024
@starocean999@dataroaring
[fix](nereids)support group_concat with distinct and order by (#38080)
7dd0233
This pr support group_concat with distinct and order by. So we can
support sql like bellow:
```
select group_concat(distinct x order by y) from t;
select multi_distinct_group_concat(x order by y) from t;
```
dataroaring pushed a commit that referenced this pull request Aug 16, 2024
@starocean999@dataroaring
[fix](nereids)support group_concat with distinct and order by (#38080)
0d2bef8
This pr support group_concat with distinct and order by. So we can
support sql like bellow:
```
select group_concat(distinct x order by y) from t;
select multi_distinct_group_concat(x order by y) from t;
```
@yiguoleiyiguolei mentioned this pull request Sep 5, 2024
Closed
3 tasks
@gavinchougavinchou mentioned this pull request Oct 13, 2024
Closed
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@starocean999@HappenLee
[fix](nereids)support group_concat with distinct and order by (apache…
fa3e8c3
…#38080)
This pr support group_concat with distinct and order by. So we can
support sql like bellow:
```
select group_concat(distinct x order by y) from t;
select multi_distinct_group_concat(x order by y) from t;
```
This was referenced May 23, 2026
Merged
Closed
morningman pushed a commit to apache/doris-website that referenced this pull request May 23, 2026
@boluor@claude
[fix](doc) add GROUP_CONCAT ORDER BY examples to 2.1/3.x group-concat…
b623e19
… docs (#3744)
## Summary
The `GROUP_CONCAT` page syntax already advertises `[ORDER BY
{<col_name>|<expr>} [ASC|DESC]]`, but the Examples section on 2.1/3.x
only showed the basic and `DISTINCT` forms, so a reader couldn't see how
`ORDER BY` actually attaches *inside* `GROUP_CONCAT`. The current/4.x
pages already carry two `ORDER BY` examples — backport them to 2.1/3.x
(EN + zh), with a version note on the DISTINCT + ORDER BY combination so
users on older patch releases don't hit a parser error.
Added examples (matching the form already shipped in current/4.x):
```sql
select GROUP_CONCAT(value ORDER BY value DESC) from test;
-- => "c, c, b, a"
-- DISTINCT + ORDER BY is supported since 2.1.6 / 3.0.2 — see note above the example
select GROUP_CONCAT(DISTINCT value ORDER BY value DESC) from test;
-- => "c, b, a"
```
### Version verification (from Doris source)
- Basic `GROUP_CONCAT(... ORDER BY ...)`: introduced in Nereids by
apache/doris#15671 + apache/doris#16081 (Jan 2023, before 2.0 GA) →
works on 2.1.x and 3.x.
- `GROUP_CONCAT(DISTINCT ... ORDER BY ...)`: apache/doris#38080, labels
`dev/2.1.6-merged, dev/3.0.2-merged` → only valid on 2.1.6+ and 3.0.2+.
The page therefore carries an inline `:::note` flagging this
minimum-version requirement (2.1.6 in the 2.1 docs, 3.0.2 in the 3.x
docs).
## Test plan
- [x] All 4 backported files now have 6 `GROUP_CONCAT(...)` example
calls, matching current/4.x.
- [x] DISTINCT + ORDER BY example is accompanied by a minimum-version
note (2.1.6 for the 2.1 docs, 3.0.2 for the 3.x docs).
- [x] Dead-link check passes.
- [ ] CI build.
Closes#3487
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BiteTheDDDDtBiteTheDDDDtBiteTheDDDDt approved these changes
@924060929924060929924060929 approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.1.6-mergeddev/3.0.2-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@starocean999@doris-robot@BiteTheDDDDt@924060929@yiguolei@dataroaring