[feat](Nereids) support set var in hint when parse sql by LiBinfeng-01 · Pull Request #41331 · apache/doris

LiBinfeng-01 · 2024-09-26T03:47:16Z

set var hint need to be enable to use before analyze, so it need to be set when parsing sql
now it would set twice when parse and begin of analyze

doris-robot · 2024-09-26T03:47:21Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

LiBinfeng-01 · 2024-09-26T05:07:11Z

run buildall

doris-robot · 2024-09-26T06:08:45Z

TPC-H: Total hot run time: 40966 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a6aca99b5dcb38fe119328f3918aa3f59b629c38, data reload: false
------ Round 1 ----------------------------------
q1	17567	7545	7339	7339
q2	2014	298	272	272
q3	12256	1042	1202	1042
q4	10566	739	715	715
q5	7808	2986	2828	2828
q6	236	149	145	145
q7	993	605	615	605
q8	9579	1965	1961	1961
q9	8153	6532	6429	6429
q10	6973	2286	2315	2286
q11	438	246	242	242
q12	409	217	214	214
q13	17784	3011	2961	2961
q14	241	217	215	215
q15	586	545	531	531
q16	693	617	596	596
q17	1000	495	588	495
q18	7258	6647	6747	6647
q19	1400	1109	1158	1109
q20	465	210	195	195
q21	4159	3228	3153	3153
q22	1115	997	986	986
Total cold run time: 111693 ms
Total hot run time: 40966 ms
----- Round 2, with runtime_filter_mode=off -----
q1	7283	7233	7301	7233
q2	339	237	232	232
q3	3041	3030	2932	2932
q4	2053	1794	1776	1776
q5	5735	5671	5761	5671
q6	236	141	139	139
q7	2246	1927	1848	1848
q8	3390	3582	3517	3517
q9	9035	9038	9006	9006
q10	3555	3590	3566	3566
q11	596	494	474	474
q12	800	630	621	621
q13	8164	3146	3167	3146
q14	299	282	270	270
q15	573	529	528	528
q16	713	698	656	656
q17	1875	1660	1601	1601
q18	8359	7827	7772	7772
q19	1747	1574	1555	1555
q20	2139	1900	1899	1899
q21	5582	5270	5503	5270
q22	1131	1049	1041	1041
Total cold run time: 68891 ms
Total hot run time: 60753 ms

doris-robot · 2024-09-26T06:20:55Z

TPC-DS: Total hot run time: 193121 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a6aca99b5dcb38fe119328f3918aa3f59b629c38, data reload: false
query1	947	382	426	382
query2	6395	2107	2060	2060
query3	8673	199	204	199
query4	34096	23508	23495	23495
query5	3375	460	462	460
query6	273	179	182	179
query7	4206	312	332	312
query8	307	233	231	231
query9	9671	2667	2666	2666
query10	434	282	274	274
query11	17937	15205	15153	15153
query12	152	101	99	99
query13	1541	430	446	430
query14	9186	7455	7495	7455
query15	291	177	176	176
query16	8025	482	495	482
query17	1701	626	584	584
query18	2161	318	334	318
query19	371	169	155	155
query20	124	115	117	115
query21	209	113	113	113
query22	5102	4851	4745	4745
query23	35336	34545	34536	34536
query24	11174	2903	2787	2787
query25	617	415	411	411
query26	1130	168	160	160
query27	2402	302	295	295
query28	7748	2454	2471	2454
query29	815	436	427	427
query30	258	158	154	154
query31	1014	807	795	795
query32	97	54	53	53
query33	746	300	292	292
query34	923	503	494	494
query35	857	733	727	727
query36	1100	937	936	936
query37	149	84	85	84
query38	3993	3880	3912	3880
query39	1495	1437	1433	1433
query40	208	97	99	97
query41	52	49	51	49
query42	117	98	99	98
query43	544	492	485	485
query44	1248	808	796	796
query45	196	163	162	162
query46	1145	722	708	708
query47	1939	1808	1844	1808
query48	456	367	384	367
query49	911	399	380	380
query50	837	411	403	403
query51	7035	6826	6859	6826
query52	97	86	84	84
query53	257	181	180	180
query54	1091	469	457	457
query55	72	71	73	71
query56	315	270	260	260
query57	1249	1094	1133	1094
query58	233	237	240	237
query59	3218	3013	3262	3013
query60	296	268	266	266
query61	103	102	103	102
query62	833	653	674	653
query63	221	183	183	183
query64	3948	649	623	623
query65	3278	3220	3205	3205
query66	817	308	312	308
query67	16051	15591	15826	15591
query68	4884	581	554	554
query69	540	299	302	299
query70	1190	1151	1101	1101
query71	393	267	283	267
query72	7366	4020	3978	3978
query73	761	344	348	344
query74	10396	9035	8919	8919
query75	3478	2690	2677	2677
query76	3371	915	895	895
query77	553	294	292	292
query78	10864	10096	9720	9720
query79	2871	615	607	607
query80	1890	453	452	452
query81	592	240	236	236
query82	628	142	140	140
query83	301	137	134	134
query84	281	82	77	77
query85	1033	299	276	276
query86	489	296	314	296
query87	4460	4322	4414	4322
query88	3961	2425	2372	2372
query89	406	290	281	281
query90	2124	186	183	183
query91	201	163	150	150
query92	64	48	47	47
query93	1987	561	548	548
query94	1039	305	289	289
query95	360	249	251	249
query96	612	277	285	277
query97	3253	3199	3162	3162
query98	211	198	194	194
query99	1524	1286	1276	1276
Total cold run time: 303881 ms
Total hot run time: 193121 ms

doris-robot · 2024-09-26T06:26:24Z

ClickBench: Total hot run time: 33.05 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a6aca99b5dcb38fe119328f3918aa3f59b629c38, data reload: false
query1	0.05	0.05	0.04
query2	0.06	0.03	0.03
query3	0.23	0.07	0.06
query4	1.64	0.10	0.10
query5	0.51	0.51	0.51
query6	1.13	0.72	0.72
query7	0.03	0.01	0.02
query8	0.03	0.03	0.04
query9	0.57	0.50	0.51
query10	0.55	0.55	0.56
query11	0.13	0.10	0.11
query12	0.14	0.10	0.11
query13	0.60	0.59	0.61
query14	2.99	3.03	3.11
query15	0.90	0.81	0.82
query16	0.40	0.38	0.39
query17	1.07	1.05	1.06
query18	0.19	0.19	0.20
query19	1.88	1.85	2.03
query20	0.01	0.01	0.01
query21	15.38	0.61	0.60
query22	3.20	2.55	1.52
query23	17.37	0.92	0.86
query24	3.37	1.20	1.47
query25	0.20	0.18	0.18
query26	0.45	0.14	0.14
query27	0.04	0.06	0.03
query28	9.99	1.10	1.07
query29	12.51	3.22	3.19
query30	0.25	0.06	0.06
query31	2.87	0.38	0.37
query32	3.29	0.48	0.46
query33	2.94	3.02	3.09
query34	16.96	4.51	4.46
query35	4.50	4.46	4.48
query36	0.64	0.50	0.48
query37	0.08	0.07	0.06
query38	0.04	0.04	0.03
query39	0.02	0.02	0.02
query40	0.15	0.13	0.13
query41	0.07	0.02	0.02
query42	0.03	0.02	0.01
query43	0.04	0.02	0.02
Total cold run time: 107.5 s
Total hot run time: 33.05 s

github-actions · 2024-09-26T08:21:03Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-09-26T08:21:06Z

PR approved by anyone and no changes requested.

LiBinfeng-01 · 2024-09-27T03:23:29Z

run p0

LiBinfeng-01 · 2024-09-27T03:23:36Z

run cloud_p0

LiBinfeng-01 · 2024-09-27T06:49:54Z

run p0

LiBinfeng-01 · 2024-09-27T06:50:01Z

run cloud_p0

LiBinfeng-01 · 2024-09-27T08:18:42Z

run cloud_p0

morrySnow · 2024-09-27T09:59:38Z

run cloud_p0

LiBinfeng-01 · 2024-09-27T11:39:36Z

run cloud_p0

LiBinfeng-01 · 2024-09-27T13:01:40Z

run cloud_p0

LiBinfeng-01 · 2024-10-08T02:31:41Z

run buildall

doris-robot · 2024-10-08T03:44:54Z

TPC-H: Total hot run time: 40768 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a6aca99b5dcb38fe119328f3918aa3f59b629c38, data reload: false
------ Round 1 ----------------------------------
q1	17642	7514	7296	7296
q2	2018	289	276	276
q3	12145	1060	1204	1060
q4	10575	776	746	746
q5	7775	2880	2904	2880
q6	238	149	153	149
q7	1026	623	625	623
q8	9359	1929	1916	1916
q9	6581	6457	6446	6446
q10	6984	2294	2368	2294
q11	444	256	250	250
q12	423	222	212	212
q13	17797	3000	3003	3000
q14	257	209	207	207
q15	565	511	521	511
q16	653	580	585	580
q17	982	620	542	542
q18	7424	6735	6637	6637
q19	1343	1028	942	942
q20	505	212	205	205
q21	3961	3322	3026	3026
q22	1120	970	985	970
Total cold run time: 109817 ms
Total hot run time: 40768 ms
----- Round 2, with runtime_filter_mode=off -----
q1	7310	7279	7236	7236
q2	325	232	230	230
q3	3055	2956	2964	2956
q4	2083	1863	1811	1811
q5	5810	5770	5785	5770
q6	246	146	149	146
q7	2295	1893	1837	1837
q8	3500	3670	3530	3530
q9	9085	8969	8954	8954
q10	3620	3562	3539	3539
q11	589	507	508	507
q12	864	642	635	635
q13	9110	3253	3218	3218
q14	298	293	271	271
q15	585	525	522	522
q16	703	650	633	633
q17	1893	1647	1612	1612
q18	8427	7754	7544	7544
q19	1720	1477	1446	1446
q20	2138	1891	1914	1891
q21	5646	5578	5567	5567
q22	1111	1075	1110	1075
Total cold run time: 70413 ms
Total hot run time: 60930 ms

doris-robot · 2024-10-08T03:56:56Z

TPC-DS: Total hot run time: 192246 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a6aca99b5dcb38fe119328f3918aa3f59b629c38, data reload: false
query1	913	396	393	393
query2	6251	2052	2077	2052
query3	8688	194	196	194
query4	34253	23668	23479	23479
query5	3414	474	467	467
query6	270	162	163	162
query7	4190	312	296	296
query8	281	224	215	215
query9	9121	2666	2666	2666
query10	443	291	298	291
query11	18008	15170	15284	15170
query12	154	97	100	97
query13	1573	470	441	441
query14	9058	7076	7418	7076
query15	253	168	177	168
query16	8061	480	470	470
query17	1689	612	607	607
query18	2150	338	321	321
query19	374	166	156	156
query20	124	120	119	119
query21	213	111	105	105
query22	4824	4716	4722	4716
query23	35064	34100	34357	34100
query24	11089	2870	2940	2870
query25	642	411	395	395
query26	1122	164	163	163
query27	2553	300	297	297
query28	7625	2407	2396	2396
query29	821	444	437	437
query30	291	154	152	152
query31	1050	807	828	807
query32	99	58	54	54
query33	768	304	298	298
query34	927	508	527	508
query35	894	738	729	729
query36	1113	935	942	935
query37	156	90	93	90
query38	3984	3932	3955	3932
query39	1484	1445	1418	1418
query40	213	99	98	98
query41	48	45	44	44
query42	124	95	98	95
query43	518	492	472	472
query44	1272	796	815	796
query45	200	162	169	162
query46	1179	717	721	717
query47	1927	1864	1812	1812
query48	434	368	342	342
query49	946	409	413	409
query50	836	420	420	420
query51	7044	6840	6803	6803
query52	101	89	89	89
query53	258	181	187	181
query54	1135	479	477	477
query55	81	78	77	77
query56	291	272	299	272
query57	1232	1143	1150	1143
query58	230	237	224	224
query59	3077	3011	2919	2919
query60	297	270	290	270
query61	105	104	99	99
query62	856	663	656	656
query63	213	185	182	182
query64	3960	636	622	622
query65	3234	3189	3213	3189
query66	828	301	302	301
query67	15853	15671	15599	15599
query68	4334	561	561	561
query69	457	292	293	292
query70	1230	1075	1119	1075
query71	346	270	274	270
query72	7150	4029	3960	3960
query73	780	347	354	347
query74	10133	9056	9064	9056
query75	3425	2724	2665	2665
query76	2564	864	936	864
query77	632	296	294	294
query78	10472	9609	9643	9609
query79	1150	587	593	587
query80	2209	442	449	442
query81	568	244	244	244
query82	761	145	138	138
query83	289	139	142	139
query84	272	81	80	80
query85	1533	300	293	293
query86	397	310	297	297
query87	4507	4270	4219	4219
query88	3380	2448	2433	2433
query89	408	292	292	292
query90	2238	191	187	187
query91	156	115	107	107
query92	63	50	46	46
query93	1078	557	546	546
query94	1160	301	299	299
query95	354	259	252	252
query96	616	281	288	281
query97	3360	3161	3176	3161
query98	216	194	198	194
query99	1561	1313	1304	1304
Total cold run time: 298278 ms
Total hot run time: 192246 ms

doris-robot · 2024-10-08T04:02:31Z

ClickBench: Total hot run time: 33.11 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a6aca99b5dcb38fe119328f3918aa3f59b629c38, data reload: false
query1	0.05	0.04	0.04
query2	0.06	0.03	0.03
query3	0.23	0.07	0.06
query4	1.64	0.10	0.11
query5	0.53	0.51	0.50
query6	1.14	0.73	0.73
query7	0.02	0.01	0.01
query8	0.04	0.04	0.03
query9	0.57	0.50	0.49
query10	0.54	0.58	0.54
query11	0.14	0.10	0.10
query12	0.14	0.10	0.10
query13	0.60	0.59	0.59
query14	2.76	2.71	2.72
query15	0.90	0.84	0.83
query16	0.38	0.40	0.39
query17	0.97	1.07	1.04
query18	0.20	0.19	0.20
query19	1.97	1.88	2.02
query20	0.01	0.01	0.01
query21	15.36	0.58	0.60
query22	2.60	2.56	1.84
query23	16.82	1.24	0.84
query24	2.78	1.16	1.24
query25	0.32	0.07	0.05
query26	0.43	0.14	0.13
query27	0.05	0.04	0.05
query28	10.77	1.09	1.07
query29	12.63	3.36	3.41
query30	0.25	0.06	0.05
query31	3.19	0.39	0.40
query32	3.34	0.47	0.46
query33	3.02	3.01	3.00
query34	17.03	4.49	4.52
query35	4.54	4.55	4.51
query36	0.68	0.48	0.48
query37	0.09	0.05	0.06
query38	0.05	0.03	0.04
query39	0.04	0.02	0.02
query40	0.15	0.13	0.12
query41	0.08	0.02	0.03
query42	0.03	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 107.18 s
Total hot run time: 33.11 s

LiBinfeng-01 · 2024-10-08T06:35:18Z

run compile

LiBinfeng-01 · 2024-10-08T06:35:26Z

run cloud_p0

LiBinfeng-01 · 2024-10-09T08:37:23Z

run p0

LiBinfeng-01 · 2024-10-10T02:39:42Z

run p0

set var hint need to be enable to use before analyze, so it need to be set when parsing sql now it would set twice when parse and begin of analyze

…#41164)(#41331)(#41546) (#41838) cherry-pick: #39723 #41164 #41331 #41546 because later problem is intro by prev one, so put them together when using fold constant by be, the return type of substring('123456',1, 3) would changed to be text, which we want it to be 3 remove windowframe in window expression to avoid folding constant on be

set var hint need to be enable to use before analyze, so it need to be set when parsing sql now it would set twice when parse and begin of analyze

pick: #41331 set var hint need to be enable to use before analyze, so it need to be set when parsing sql now it would set twice when parse and begin of analyze ## Proposed changes Issue Number: close #xxx

set var hint need to be enable to use before analyze, so it need to be set when parsing sql now it would set twice when parse and begin of analyze

[feat](Nereids) support set var in hint when parse sql
a6aca99

morrySnow added dev/2.0.x dev/2.1.x labels Sep 26, 2024

englefly approved these changes Sep 26, 2024
View reviewed changes

github-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Sep 26, 2024

924060929 approved these changes Sep 26, 2024
View reviewed changes

924060929 merged commit 4583a02 into apache:masterOct 10, 2024

LiBinfeng-01 mentioned this pull request Oct 15, 2024
[feat](Nereids) support set var in hint when parse sql (#41331) #41863
Closed

This was referenced Oct 17, 2024
[Feat](Nereids) support use cbo rule hint #35925 #39715 #40167#40958 #41869
Merged
[fix](Nereids) fix fold constant by be return type mismatched (#39723)(#41164)(#41331)(#41546) #41838
Merged

924060929 added dev/2.1.7-merged and removed dev/2.1.x labels Oct 18, 2024

LiBinfeng-01 mentioned this pull request Oct 23, 2024
[feat](Nereids) support set var in hint when parse sql (#41331) #42335
Merged

starocean999 added dev/2.0.16-merged-deprecated 2.0.16 is NOT and will NOT be released and removed dev/2.0.x labels Oct 25, 2024

Uh oh!

Conversation

LiBinfeng-01 commented Sep 26, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

doris-robot commented Sep 26, 2024

Uh oh!

LiBinfeng-01 commented Sep 26, 2024

Uh oh!

doris-robot commented Sep 26, 2024

Uh oh!

doris-robot commented Sep 26, 2024

Uh oh!

doris-robot commented Sep 26, 2024

Uh oh!

github-actionsBot commented Sep 26, 2024

Uh oh!

github-actionsBot commented Sep 26, 2024

Uh oh!

LiBinfeng-01 commented Sep 27, 2024

Uh oh!

LiBinfeng-01 commented Sep 27, 2024

Uh oh!

LiBinfeng-01 commented Sep 27, 2024

Uh oh!

LiBinfeng-01 commented Sep 27, 2024

Uh oh!

LiBinfeng-01 commented Sep 27, 2024

Uh oh!

morrySnow commented Sep 27, 2024

Uh oh!

LiBinfeng-01 commented Sep 27, 2024

Uh oh!

LiBinfeng-01 commented Sep 27, 2024

Uh oh!

LiBinfeng-01 commented Oct 8, 2024

Uh oh!

doris-robot commented Oct 8, 2024

Uh oh!

doris-robot commented Oct 8, 2024

Uh oh!

doris-robot commented Oct 8, 2024

Uh oh!

LiBinfeng-01 commented Oct 8, 2024

Uh oh!

LiBinfeng-01 commented Oct 8, 2024

Uh oh!

LiBinfeng-01 commented Oct 9, 2024

Uh oh!

LiBinfeng-01 commented Oct 10, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

LiBinfeng-01 commented Sep 26, 2024•
edited
Loading