[fix](Nereids) fix lexer Backtracking or Ambiguity cause of key word duplicate by LiBinfeng-01 · Pull Request #39590 · apache/doris

LiBinfeng-01 · 2024-08-20T02:04:00Z

nereids timeout cause of parser cost too long time
intro by： #39113

Proposed changes

Issue Number: close #xxx

doris-robot · 2024-08-20T02:04:06Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

LiBinfeng-01 · 2024-08-20T02:04:21Z

run buildall

doris-robot · 2024-08-20T02:51:07Z

TPC-H: Total hot run time: 37610 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d894a74acaff3ff07937c16eba5e78541dca2f5f, data reload: false
------ Round 1 ----------------------------------
q1	17696	4387	4281	4281
q2	2024	177	172	172
q3	11862	973	1102	973
q4	10489	708	802	708
q5	7752	2802	2817	2802
q6	225	139	137	137
q7	966	608	616	608
q8	9534	2056	2079	2056
q9	8696	6627	6534	6534
q10	7002	2191	2211	2191
q11	445	244	243	243
q12	391	224	230	224
q13	18566	2994	3013	2994
q14	271	234	245	234
q15	518	478	483	478
q16	497	398	388	388
q17	997	632	707	632
q18	7441	6646	6636	6636
q19	7281	980	1051	980
q20	677	334	323	323
q21	4066	3077	3033	3033
q22	1089	1003	983	983
Total cold run time: 118485 ms
Total hot run time: 37610 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4457	4291	4267	4267
q2	363	280	268	268
q3	2815	2635	2638	2635
q4	2016	1778	1724	1724
q5	5675	5729	5653	5653
q6	231	133	137	133
q7	2093	1747	1841	1747
q8	3381	3479	3406	3406
q9	8810	8887	8920	8887
q10	3478	3261	3340	3261
q11	603	507	499	499
q12	782	613	606	606
q13	16960	3123	3161	3123
q14	306	292	273	273
q15	545	507	481	481
q16	516	428	453	428
q17	1848	1542	1516	1516
q18	8050	7782	7770	7770
q19	1721	1571	1544	1544
q20	2173	1887	1901	1887
q21	8483	5464	5388	5388
q22	1092	1055	1027	1027
Total cold run time: 76398 ms
Total hot run time: 56523 ms

morrySnow · 2024-08-20T03:00:42Z


 BRACKETED_COMMENT
- : '/*' {!isHint()}? ( BRACKETED_COMMENT | . )*? ('*/' | {markUnclosedComment();} EOF) -> channel(HIDDEN)
+ : COMMENT_START ( BRACKETED_COMMENT | . )*? ('*/' | {markUnclosedComment();} EOF) -> channel(HIDDEN)


Suggested change
: COMMENT_START ( BRACKETED_COMMENT | . )*? ('*/' | {markUnclosedComment();}EOF) -> channel(HIDDEN)
: COMMENT_START ( BRACKETED_COMMENT | HINT_WITH_CHANNEL | . )*? ('*/' | {markUnclosedComment();}EOF) -> channel(HIDDEN)

morrySnow · 2024-08-20T03:01:32Z

 ARROW: '->';
 HINT_START: '/*+';
 HINT_END: '*/';
+COMMENT_START: '/*';


add to non-reserved key word

morrySnow · 2024-08-20T03:01:50Z


 HINT_WITH_CHANNEL
- : {isChannel2()}? HINT_START .*? HINT_END -> channel(2)
+ : HINT_START {isChannel2()}? .*? HINT_END -> channel(2)


Suggested change
: HINT_START{isChannel2()}? .*? HINT_END -> channel(2)
: HINT_START{isChannel2()}? ( BRACKETED_COMMENT | HINT_WITH_CHANNEL | . )*? HINT_END -> channel(2)

morrySnow · 2024-08-20T03:03:06Z

+ : COMMENT_START ( BRACKETED_COMMENT | . )*? ('*/' | {markUnclosedComment();} EOF) -> channel(HIDDEN)
 ;

 HINT_WITH_CHANNEL


Suggested change
HINT_WITH_CHANNEL
HINT

LiBinfeng-01 · 2024-08-20T03:45:17Z

run buildall

LiBinfeng-01 · 2024-08-20T04:02:20Z

run buildall

morrySnow · 2024-08-20T06:23:59Z

 | COLOCATE
 | COLUMNS
 | COMMENT
+ | COMMENT_START


add HINT_START HINT_END too

LiBinfeng-01 · 2024-08-20T07:18:44Z

run buildall

doris-robot · 2024-08-20T09:45:03Z

TPC-H: Total hot run time: 37419 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5984622c159fe6c2377dfa5ece224e9a97d00b01, data reload: false
------ Round 1 ----------------------------------
q1	17638	4403	4258	4258
q2	2016	176	167	167
q3	11843	974	1126	974
q4	10526	746	799	746
q5	7766	2826	2678	2678
q6	221	137	136	136
q7	981	609	585	585
q8	9552	2020	2064	2020
q9	8672	6495	6522	6495
q10	7015	2176	2166	2166
q11	444	239	237	237
q12	410	232	225	225
q13	17781	3001	2997	2997
q14	281	243	238	238
q15	519	486	497	486
q16	499	402	387	387
q17	979	625	679	625
q18	7369	6728	6740	6728
q19	6712	1132	1094	1094
q20	673	325	311	311
q21	3813	2902	2887	2887
q22	1086	979	994	979
Total cold run time: 116796 ms
Total hot run time: 37419 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4440	4288	4272	4272
q2	374	280	280	280
q3	2886	2672	2658	2658
q4	1969	1711	1715	1711
q5	5715	5856	5624	5624
q6	235	135	133	133
q7	2124	1752	1760	1752
q8	3311	3512	3446	3446
q9	8812	8712	8906	8712
q10	3540	3247	3327	3247
q11	605	508	511	508
q12	803	625	602	602
q13	16636	3169	3192	3169
q14	329	279	301	279
q15	523	497	489	489
q16	500	445	448	445
q17	1830	1540	1547	1540
q18	8119	8034	7787	7787
q19	1711	1506	1637	1506
q20	2099	1901	1848	1848
q21	9753	5386	5404	5386
q22	1099	1025	1026	1025
Total cold run time: 77413 ms
Total hot run time: 56419 ms

doris-robot · 2024-08-20T09:57:40Z

TPC-DS: Total hot run time: 189113 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5984622c159fe6c2377dfa5ece224e9a97d00b01, data reload: false
query1	1233	877	876	876
query2	6435	1905	1844	1844
query3	10606	3937	3785	3785
query4	59722	23674	22957	22957
query5	6082	494	492	492
query6	502	161	150	150
query7	6509	299	293	293
query8	307	210	196	196
query9	8890	2432	2398	2398
query10	515	274	251	251
query11	18180	14970	15225	14970
query12	163	104	104	104
query13	1608	370	394	370
query14	12042	7193	6891	6891
query15	290	170	181	170
query16	7786	515	464	464
query17	1398	583	558	558
query18	2133	294	291	291
query19	288	145	153	145
query20	133	117	107	107
query21	210	102	98	98
query22	4608	4402	4320	4320
query23	34302	33501	33255	33255
query24	5628	2828	2847	2828
query25	557	403	399	399
query26	684	158	157	157
query27	1811	278	275	275
query28	3834	2049	2025	2025
query29	732	420	423	420
query30	304	155	152	152
query31	941	759	740	740
query32	101	56	55	55
query33	495	285	283	283
query34	859	467	451	451
query35	796	702	730	702
query36	1085	905	911	905
query37	139	82	82	82
query38	3896	3884	3749	3749
query39	1435	1391	1410	1391
query40	202	119	119	119
query41	49	47	47	47
query42	116	95	95	95
query43	505	464	475	464
query44	1067	733	744	733
query45	197	166	164	164
query46	1088	723	709	709
query47	1874	1772	1762	1762
query48	370	293	284	284
query49	794	427	434	427
query50	799	421	403	403
query51	7132	6930	6964	6930
query52	103	87	92	87
query53	261	180	183	180
query54	562	459	459	459
query55	80	77	76	76
query56	282	253	252	252
query57	1172	1078	1067	1067
query58	235	230	230	230
query59	2906	2631	2817	2631
query60	293	265	278	265
query61	122	116	117	116
query62	750	615	622	615
query63	213	181	180	180
query64	4299	2342	1786	1786
query65	3213	3169	3169	3169
query66	725	338	351	338
query67	15234	14909	14728	14728
query68	7193	555	553	553
query69	597	378	293	293
query70	1169	1094	1054	1054
query71	503	278	275	275
query72	6860	2302	2008	2008
query73	780	315	319	315
query74	9202	8739	8636	8636
query75	3840	2681	2657	2657
query76	4234	965	975	965
query77	697	320	308	308
query78	9917	9038	9527	9038
query79	11892	528	536	528
query80	1630	519	488	488
query81	576	222	223	222
query82	716	136	132	132
query83	281	147	145	145
query84	266	75	77	75
query85	1159	325	282	282
query86	389	287	295	287
query87	4456	4172	4225	4172
query88	4616	2386	2303	2303
query89	443	286	282	282
query90	1956	193	193	193
query91	127	99	111	99
query92	63	51	52	51
query93	5228	530	529	529
query94	730	281	286	281
query95	343	259	260	259
query96	607	270	268	268
query97	3189	3078	3019	3019
query98	212	209	208	208
query99	1681	1298	1273	1273
Total cold run time: 337434 ms
Total hot run time: 189113 ms

doris-robot · 2024-08-20T10:03:11Z

ClickBench: Total hot run time: 31.45 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5984622c159fe6c2377dfa5ece224e9a97d00b01, data reload: false
query1	0.05	0.05	0.04
query2	0.07	0.04	0.04
query3	0.22	0.06	0.06
query4	1.66	0.08	0.08
query5	0.50	0.49	0.49
query6	1.13	0.72	0.72
query7	0.02	0.02	0.01
query8	0.06	0.05	0.05
query9	0.55	0.50	0.48
query10	0.54	0.55	0.54
query11	0.15	0.11	0.11
query12	0.14	0.12	0.13
query13	0.61	0.60	0.58
query14	0.75	0.78	0.78
query15	0.84	0.82	0.81
query16	0.36	0.36	0.38
query17	1.04	1.05	1.08
query18	0.22	0.19	0.21
query19	1.92	1.75	1.80
query20	0.01	0.01	0.02
query21	15.39	0.68	0.67
query22	3.29	7.24	2.75
query23	18.70	1.46	1.29
query24	2.13	0.22	0.23
query25	0.15	0.08	0.08
query26	0.27	0.17	0.18
query27	0.08	0.08	0.07
query28	13.20	1.02	1.01
query29	12.63	3.32	3.28
query30	0.24	0.06	0.06
query31	2.89	0.40	0.39
query32	3.26	0.47	0.48
query33	2.92	2.96	2.99
query34	17.12	4.37	4.31
query35	4.45	4.41	4.39
query36	0.66	0.47	0.46
query37	0.19	0.17	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.03
query40	0.16	0.13	0.12
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.05
Total cold run time: 108.98 s
Total hot run time: 31.45 s

LiBinfeng-01 · 2024-08-20T11:27:05Z

run buildall

doris-robot · 2024-08-20T14:34:37Z

TPC-H: Total hot run time: 37920 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6a222f3ad3272b52f2c242fb68601c049998fb48, data reload: false
------ Round 1 ----------------------------------
q1	17745	4345	4288	4288
q2	2045	195	190	190
q3	11876	995	1069	995
q4	10506	754	738	738
q5	7748	2851	2766	2766
q6	240	147	144	144
q7	1013	624	613	613
q8	9583	2088	2064	2064
q9	8317	6518	6538	6518
q10	7027	2219	2177	2177
q11	459	252	260	252
q12	414	241	246	241
q13	19135	3019	3008	3008
q14	287	251	243	243
q15	544	501	517	501
q16	511	416	402	402
q17	987	722	770	722
q18	7351	6822	6866	6822
q19	7092	1045	1083	1045
q20	704	348	341	341
q21	3822	3014	2839	2839
q22	1097	1011	1012	1011
Total cold run time: 118503 ms
Total hot run time: 37920 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4463	4293	4285	4285
q2	394	304	294	294
q3	2836	2586	2684	2586
q4	1983	1786	1728	1728
q5	5621	5729	5660	5660
q6	233	139	144	139
q7	2177	1794	1812	1794
q8	3343	3483	3424	3424
q9	8763	8724	8787	8724
q10	3528	3331	3291	3291
q11	588	521	515	515
q12	840	689	654	654
q13	16251	3109	3145	3109
q14	315	295	287	287
q15	558	519	509	509
q16	522	453	461	453
q17	1816	1556	1534	1534
q18	8324	7898	7872	7872
q19	6944	1449	1585	1449
q20	2183	1885	1893	1885
q21	14829	5224	5262	5224
q22	1124	1078	1036	1036
Total cold run time: 87635 ms
Total hot run time: 56452 ms

doris-robot · 2024-08-20T14:47:18Z

TPC-DS: Total hot run time: 193830 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6a222f3ad3272b52f2c242fb68601c049998fb48, data reload: false
query1	1284	893	871	871
query2	6589	2018	1925	1925
query3	10637	3725	3834	3725
query4	57781	25016	23253	23253
query5	5609	589	570	570
query6	468	180	177	177
query7	6057	305	297	297
query8	379	291	289	289
query9	8892	2533	2466	2466
query10	566	315	301	301
query11	18052	15232	15254	15232
query12	176	117	121	117
query13	1611	402	424	402
query14	11316	7087	7303	7087
query15	294	200	191	191
query16	7631	517	494	494
query17	1210	593	619	593
query18	2119	340	329	329
query19	307	163	158	158
query20	139	132	129	129
query21	231	123	119	119
query22	4668	4387	4288	4288
query23	33969	33907	33947	33907
query24	5960	2944	2951	2944
query25	581	412	428	412
query26	712	171	175	171
query27	1945	290	298	290
query28	3712	2112	2105	2105
query29	705	423	417	417
query30	207	167	169	167
query31	959	797	786	786
query32	87	71	68	68
query33	475	317	306	306
query34	860	478	468	468
query35	858	732	745	732
query36	1089	946	951	946
query37	145	91	90	90
query38	3936	3866	3899	3866
query39	1512	1478	1415	1415
query40	231	133	147	133
query41	101	95	96	95
query42	125	109	113	109
query43	531	509	506	506
query44	1093	768	761	761
query45	220	182	181	181
query46	1103	752	760	752
query47	1873	1777	1799	1777
query48	392	321	318	318
query49	830	509	497	497
query50	839	456	436	436
query51	7308	7063	6995	6995
query52	111	102	99	99
query53	289	212	206	206
query54	593	492	483	483
query55	84	85	83	83
query56	310	286	276	276
query57	1208	1077	1114	1077
query58	261	257	265	257
query59	2863	2822	2797	2797
query60	314	301	303	301
query61	128	128	126	126
query62	753	680	684	680
query63	239	207	209	207
query64	3330	1822	1775	1775
query65	3271	3171	3191	3171
query66	866	511	517	511
query67	15350	14997	14840	14840
query68	6688	574	572	572
query69	537	298	309	298
query70	1171	1176	1154	1154
query71	521	296	290	290
query72	6671	2332	2095	2095
query73	828	335	337	335
query74	9314	8934	8854	8854
query75	3891	2663	2684	2663
query76	3907	1062	987	987
query77	626	392	379	379
query78	9855	9095	8978	8978
query79	8051	557	546	546
query80	2004	566	567	566
query81	574	251	249	249
query82	1026	150	150	150
query83	326	182	186	182
query84	281	88	87	87
query85	1170	340	329	329
query86	447	318	314	314
query87	4311	4205	4215	4205
query88	4671	2405	2398	2398
query89	555	307	308	307
query90	2087	217	218	217
query91	140	134	114	114
query92	78	63	119	63
query93	6259	545	555	545
query94	828	303	301	301
query95	380	279	287	279
query96	613	278	276	276
query97	3213	3043	3072	3043
query98	243	215	224	215
query99	1802	1287	1301	1287
Total cold run time: 331412 ms
Total hot run time: 193830 ms

doris-robot · 2024-08-20T14:52:50Z

ClickBench: Total hot run time: 30.75 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6a222f3ad3272b52f2c242fb68601c049998fb48, data reload: false
query1	0.04	0.04	0.04
query2	0.09	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.08	0.09
query5	0.49	0.49	0.51
query6	1.15	0.73	0.73
query7	0.02	0.02	0.02
query8	0.05	0.05	0.05
query9	0.54	0.47	0.50
query10	0.54	0.53	0.55
query11	0.16	0.12	0.12
query12	0.15	0.13	0.12
query13	0.60	0.59	0.59
query14	0.76	0.80	0.79
query15	0.88	0.83	0.83
query16	0.37	0.36	0.38
query17	1.04	1.05	0.98
query18	0.20	0.20	0.21
query19	1.77	1.70	1.81
query20	0.01	0.02	0.01
query21	15.41	0.66	0.65
query22	4.53	7.20	1.82
query23	18.32	1.40	1.34
query24	2.13	0.22	0.22
query25	0.16	0.09	0.09
query26	0.26	0.18	0.18
query27	0.08	0.08	0.09
query28	13.36	1.02	1.01
query29	12.67	3.30	3.32
query30	0.35	0.16	0.16
query31	2.80	0.40	0.39
query32	3.25	0.48	0.49
query33	2.96	3.00	2.95
query34	17.12	4.30	4.32
query35	4.42	4.48	4.42
query36	0.66	0.48	0.47
query37	0.21	0.18	0.17
query38	0.17	0.17	0.17
query39	0.06	0.05	0.05
query40	0.17	0.15	0.14
query41	0.10	0.05	0.05
query42	0.07	0.06	0.06
query43	0.07	0.05	0.05
Total cold run time: 110.09 s
Total hot run time: 30.75 s

LiBinfeng-01 · 2024-08-20T17:01:05Z

run buildall

doris-robot · 2024-08-20T19:51:10Z

TPC-H: Total hot run time: 38354 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e37b3ab0c2446a1e697dc706f7928c43e9f6ea4d, data reload: false
------ Round 1 ----------------------------------
q1	17858	4359	4267	4267
q2	2058	219	210	210
q3	11769	976	1186	976
q4	10520	804	787	787
q5	7795	2853	2804	2804
q6	265	157	156	156
q7	1011	646	654	646
q8	9586	2092	2134	2092
q9	8192	6543	6530	6530
q10	7095	2283	2321	2283
q11	512	268	267	267
q12	422	261	257	257
q13	18824	2996	2999	2996
q14	305	264	253	253
q15	569	523	535	523
q16	537	417	421	417
q17	1009	743	714	714
q18	7279	6759	6826	6759
q19	6887	1145	1157	1145
q20	741	350	360	350
q21	3878	2866	2880	2866
q22	1148	1057	1056	1056
Total cold run time: 118260 ms
Total hot run time: 38354 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4567	4305	4325	4305
q2	422	311	308	308
q3	2869	2621	2752	2621
q4	1979	1748	1729	1729
q5	5619	5706	5636	5636
q6	241	151	147	147
q7	2191	1760	1748	1748
q8	3262	3520	3457	3457
q9	8764	8784	8653	8653
q10	3616	3360	3267	3267
q11	612	529	517	517
q12	821	679	668	668
q13	16976	3183	3138	3138
q14	324	297	313	297
q15	563	508	514	508
q16	506	471	460	460
q17	1831	1527	1567	1527
q18	8213	7889	7686	7686
q19	1752	1627	1362	1362
q20	2150	1909	1907	1907
q21	14260	5318	5382	5318
q22	1196	1095	1099	1095
Total cold run time: 82734 ms
Total hot run time: 56354 ms

doris-robot · 2024-08-20T20:04:08Z

TPC-DS: Total hot run time: 195202 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e37b3ab0c2446a1e697dc706f7928c43e9f6ea4d, data reload: false
query1	1336	913	892	892
query2	6624	1980	1937	1937
query3	10665	4056	3636	3636
query4	55414	27254	23233	23233
query5	5965	711	713	711
query6	543	212	207	207
query7	6419	329	332	329
query8	544	432	434	432
query9	9252	2523	2480	2480
query10	604	348	341	341
query11	18393	15033	15373	15033
query12	198	149	140	140
query13	1699	444	446	444
query14	11947	6880	6719	6719
query15	257	207	194	194
query16	7713	504	508	504
query17	1228	618	601	601
query18	2140	352	340	340
query19	309	170	168	168
query20	155	141	137	137
query21	252	141	145	141
query22	4563	4534	4277	4277
query23	34279	33596	33810	33596
query24	5779	2988	2974	2974
query25	571	441	446	441
query26	717	202	186	186
query27	1922	301	304	301
query28	3935	2198	2150	2150
query29	744	449	467	449
query30	229	185	186	185
query31	992	828	830	828
query32	101	78	79	78
query33	523	332	347	332
query34	886	496	507	496
query35	857	773	770	770
query36	1101	960	973	960
query37	153	103	103	103
query38	3977	3878	3835	3835
query39	1556	1502	1466	1466
query40	231	153	149	149
query41	136	136	135	135
query42	134	124	114	114
query43	538	508	487	487
query44	1085	774	783	774
query45	230	198	200	198
query46	1133	794	776	776
query47	1882	1842	1878	1842
query48	408	337	331	331
query49	912	575	568	568
query50	855	459	495	459
query51	7241	7130	7033	7033
query52	117	113	107	107
query53	297	232	228	228
query54	605	510	505	505
query55	88	90	87	87
query56	323	311	305	305
query57	1176	1138	1134	1134
query58	305	329	304	304
query59	3005	2837	2761	2761
query60	365	330	324	324
query61	158	144	145	144
query62	804	710	671	671
query63	263	231	228	228
query64	4472	2353	1883	1883
query65	3233	3159	3177	3159
query66	1064	674	659	659
query67	15339	15041	14966	14966
query68	8682	601	594	594
query69	753	426	337	337
query70	1524	1058	1139	1058
query71	556	321	310	310
query72	6957	2353	2058	2058
query73	2726	356	356	356
query74	9579	8912	8934	8912
query75	4081	2719	2755	2719
query76	5045	1064	1036	1036
query77	820	432	425	425
query78	10693	9328	9001	9001
query79	9522	543	541	541
query80	1707	604	606	604
query81	596	272	256	256
query82	585	157	159	157
query83	346	209	213	209
query84	287	100	95	95
query85	867	362	354	354
query86	365	322	323	322
query87	4444	4224	4259	4224
query88	3860	2517	2535	2517
query89	551	320	318	318
query90	2158	222	224	222
query91	150	126	124	124
query92	83	74	76	74
query93	6124	552	549	549
query94	946	306	307	306
query95	387	288	292	288
query96	613	282	284	282
query97	3228	3068	3103	3068
query98	244	230	218	218
query99	1595	1291	1337	1291
Total cold run time: 341015 ms
Total hot run time: 195202 ms

doris-robot · 2024-08-20T20:09:41Z

ClickBench: Total hot run time: 31.22 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e37b3ab0c2446a1e697dc706f7928c43e9f6ea4d, data reload: false
query1	0.05	0.04	0.04
query2	0.08	0.05	0.05
query3	0.23	0.05	0.05
query4	1.67	0.08	0.10
query5	0.51	0.50	0.50
query6	1.13	0.72	0.73
query7	0.02	0.01	0.02
query8	0.05	0.05	0.05
query9	0.55	0.50	0.49
query10	0.55	0.55	0.55
query11	0.17	0.12	0.12
query12	0.16	0.13	0.14
query13	0.62	0.60	0.59
query14	0.78	0.80	0.77
query15	0.84	0.82	0.82
query16	0.38	0.35	0.35
query17	0.97	1.08	0.96
query18	0.21	0.20	0.20
query19	1.78	1.73	1.71
query20	0.01	0.02	0.01
query21	15.42	0.67	0.66
query22	4.63	6.84	2.18
query23	18.27	1.37	1.27
query24	2.14	0.23	0.23
query25	0.15	0.09	0.09
query26	0.27	0.18	0.18
query27	0.08	0.09	0.08
query28	13.20	1.03	1.01
query29	12.61	3.33	3.30
query30	0.44	0.26	0.25
query31	2.80	0.40	0.39
query32	3.25	0.49	0.48
query33	2.97	2.93	2.94
query34	17.01	4.34	4.37
query35	4.44	4.46	4.43
query36	0.67	0.50	0.49
query37	0.21	0.18	0.18
query38	0.18	0.17	0.16
query39	0.06	0.05	0.05
query40	0.17	0.15	0.14
query41	0.11	0.07	0.07
query42	0.07	0.06	0.08
query43	0.06	0.05	0.06
Total cold run time: 109.97 s
Total hot run time: 31.22 s

LiBinfeng-01 · 2024-08-21T01:57:24Z

run feut

LiBinfeng-01 · 2024-08-21T08:34:03Z

run buildall

doris-robot · 2024-08-21T09:20:14Z

TPC-H: Total hot run time: 38335 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1fb649c3f0b6259ae5b137fd3040072a4ed7e5dc, data reload: false
------ Round 1 ----------------------------------
q1	18943	4656	4293	4293
q2	2070	210	215	210
q3	10481	1136	1145	1136
q4	10210	831	757	757
q5	7757	2842	2863	2842
q6	268	159	159	159
q7	1023	657	641	641
q8	9374	2107	2115	2107
q9	7337	6587	6563	6563
q10	7098	2235	2263	2235
q11	483	274	277	274
q12	437	262	254	254
q13	17793	2992	3012	2992
q14	297	250	247	247
q15	562	515	517	515
q16	529	408	427	408
q17	1010	736	787	736
q18	7641	6955	6844	6844
q19	5258	1030	1002	1002
q20	716	360	341	341
q21	3789	3039	2742	2742
q22	1115	1037	1057	1037
Total cold run time: 114191 ms
Total hot run time: 38335 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4589	4265	4271	4265
q2	410	310	298	298
q3	2881	2624	2684	2624
q4	1897	1731	1730	1730
q5	5650	5792	5675	5675
q6	248	144	144	144
q7	2233	1791	1810	1791
q8	3335	3444	3469	3444
q9	8833	8553	8689	8553
q10	3530	3310	3303	3303
q11	620	551	537	537
q12	860	647	649	647
q13	17499	3134	3140	3134
q14	338	298	319	298
q15	575	546	575	546
q16	540	475	467	467
q17	1825	1559	1533	1533
q18	8263	7806	7729	7729
q19	8537	1612	1567	1567
q20	2069	1832	1865	1832
q21	13552	5165	5317	5165
q22	1135	1074	1063	1063
Total cold run time: 89419 ms
Total hot run time: 56345 ms

morrySnow · 2024-08-21T11:49:25Z

-HINT_WITH_CHANNEL
- : {isChannel2()}? HINT_START .*? HINT_END -> channel(2)
+BRACKETED_COMMENT
+ : {isChannel()}? COMMENT_START ( BRACKETED_COMMENT | HINT | . )*? ('*/' | {markUnclosedComment();} EOF) -> channel(HIDDEN)


why add isChannel here?

…duplicate

LiBinfeng-01 · 2024-08-21T13:57:53Z

run buildall

github-actions · 2024-08-22T02:20:52Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-08-22T02:20:54Z

PR approved by anyone and no changes requested.

…duplicate (#39590) (#39750) cherry-pick from master #39590 issue intro by #39416

…duplicate (apache#39590) nereids timeout cause of parser cost too long time intro by： apache#39113 ## Proposed changes Issue Number: close #xxx

…duplicate (#39590) (#41033) cherry-pick: #39590 nereids timeout cause of parser cost too long time intro by： #39113

…duplicate (apache#39590) (apache#39750) cherry-pick from master apache#39590 issue intro by apache#39416

…duplicate (apache#39590) nereids timeout cause of parser cost too long time intro by： apache#39113 ## Proposed changes Issue Number: close #xxx

morrySnow reviewed Aug 20, 2024
View reviewed changes

LiBinfeng-01 force-pushed the fix_lexer branch from d894a74 to b23587fCompare August 20, 2024 03:25

morrySnow reviewed Aug 20, 2024
View reviewed changes

LiBinfeng-01 force-pushed the fix_lexer branch from 5984622 to 6a222f3Compare August 20, 2024 11:26

LiBinfeng-01 force-pushed the fix_lexer branch from 6a222f3 to e37b3abCompare August 20, 2024 17:00

morrySnow reviewed Aug 21, 2024
View reviewed changes

[fix](Nereids) fix lexer Backtracking or Ambiguity cause of key word …
c62fe38
…duplicate

LiBinfeng-01 force-pushed the fix_lexer branch from 1fb649c to c62fe38Compare August 21, 2024 13:57

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Aug 22, 2024

github-actionsBot added the reviewed label Aug 22, 2024

morrySnow added dev/2.0.x dev/2.1.x p0_b labels Aug 22, 2024

LiBinfeng-01 mentioned this pull request Aug 22, 2024
[fix](Nereids) fix lexer Backtracking or Ambiguity cause of key word duplicate #39750
Merged

englefly approved these changes Aug 22, 2024
View reviewed changes

englefly merged commit 3f67494 into apache:masterAug 22, 2024

morrySnow pushed a commit that referenced this pull request Aug 22, 2024
[fix](Nereids) fix lexer Backtracking or Ambiguity cause of key word …
fdf540f
…duplicate (#39590) (#39750) cherry-pick from master #39590 issue intro by #39416

morrySnow added dev/2.0.15-merged and removed dev/2.0.x labels Aug 22, 2024

yiguolei added dev/2.1.6-merged and removed dev/2.1.x labels Aug 25, 2024

dataroaring added dev/3.0.x dev/3.0.2-merged and removed dev/3.0.x labels Aug 26, 2024

gavinchou added dev/3.0.x dev/3.0.x-conflict and removed dev/3.0.2-merged dev/3.0.x labels Sep 12, 2024

LiBinfeng-01 mentioned this pull request Sep 20, 2024
[fix](Nereids) fix lexer Backtracking or Ambiguity cause of key word duplicate (#39590) #41033
Merged

dataroaring pushed a commit that referenced this pull request Sep 20, 2024
[fix](Nereids) fix lexer Backtracking or Ambiguity cause of key word …
b4272eb
…duplicate (#39590) (#41033) cherry-pick: #39590 nereids timeout cause of parser cost too long time intro by： #39113

dataroaring added dev/3.0.2-merged and removed dev/3.0.x-conflict labels Sep 20, 2024

mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
[fix](Nereids) fix lexer Backtracking or Ambiguity cause of key word …
8a85498
…duplicate (apache#39590) (apache#39750) cherry-pick from master apache#39590 issue intro by apache#39416

	: COMMENT_START ( BRACKETED_COMMENT \| . )? ('/' \| {markUnclosedComment();}EOF) -> channel(HIDDEN)
	: COMMENT_START ( BRACKETED_COMMENT \| HINT_WITH_CHANNEL \| . )? ('/' \| {markUnclosedComment();}EOF) -> channel(HIDDEN)

	: HINT_START{isChannel2()}? .*? HINT_END -> channel(2)
	: HINT_START{isChannel2()}? ( BRACKETED_COMMENT \| HINT_WITH_CHANNEL \| . )*? HINT_END -> channel(2)

Uh oh!

Conversation

LiBinfeng-01 commented Aug 20, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Uh oh!

doris-robot commented Aug 20, 2024

Uh oh!

LiBinfeng-01 commented Aug 20, 2024

Uh oh!

doris-robot commented Aug 20, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LiBinfeng-01 commented Aug 20, 2024

Uh oh!

LiBinfeng-01 commented Aug 20, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LiBinfeng-01 commented Aug 20, 2024

Uh oh!

doris-robot commented Aug 20, 2024

Uh oh!

doris-robot commented Aug 20, 2024

Uh oh!

doris-robot commented Aug 20, 2024

Uh oh!

LiBinfeng-01 commented Aug 20, 2024

Uh oh!

doris-robot commented Aug 20, 2024

Uh oh!

doris-robot commented Aug 20, 2024

Uh oh!

doris-robot commented Aug 20, 2024

Uh oh!

LiBinfeng-01 commented Aug 20, 2024

Uh oh!

doris-robot commented Aug 20, 2024

Uh oh!

doris-robot commented Aug 20, 2024

Uh oh!

doris-robot commented Aug 20, 2024

Uh oh!

LiBinfeng-01 commented Aug 21, 2024

Uh oh!

LiBinfeng-01 commented Aug 21, 2024

Uh oh!

doris-robot commented Aug 21, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LiBinfeng-01 commented Aug 21, 2024

Uh oh!

github-actionsBot commented Aug 22, 2024

Uh oh!

github-actionsBot commented Aug 22, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

LiBinfeng-01 commented Aug 20, 2024•
edited
Loading