[fix](txn) Fix coordidator be restart not abort txn by yujun777 · Pull Request #35342 · apache/doris

yujun777 · 2024-05-24T06:59:23Z

BUG: fe will abort coordidator BE's txn when be shutdown exceeds 5min. But if BE restart within 5min, then this BE's txns will not abort until timeout.

FIX: every txn will record BE's id and its start time. When fe found be's startTime change, it will abort the BE's old txns.

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

doris-robot · 2024-05-24T06:59:28Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

github-actions · 2024-05-24T07:04:56Z

clang-tidy review says "All clean, LGTM! 👍"

yujun777 · 2024-05-24T07:12:45Z

run buildall

github-actions · 2024-05-24T07:26:11Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-05-24T07:26:14Z

PR approved by anyone and no changes requested.

doris-robot · 2024-05-24T08:13:55Z

TPC-H: Total hot run time: 40765 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0dc1c1d8e6555f1ea99b0d74c5b400832a832fa9, data reload: false
------ Round 1 ----------------------------------
q1	17797	4547	4397	4397
q2	2608	190	201	190
q3	11804	1175	1194	1175
q4	10537	815	782	782
q5	7634	2736	2707	2707
q6	224	137	136	136
q7	977	607	609	607
q8	9599	2096	2089	2089
q9	9127	6524	6463	6463
q10	8831	3692	3727	3692
q11	457	241	237	237
q12	418	225	223	223
q13	17872	2982	2952	2952
q14	257	211	219	211
q15	524	467	467	467
q16	515	388	385	385
q17	981	699	657	657
q18	8134	7586	7535	7535
q19	5067	1561	1496	1496
q20	645	300	301	300
q21	4949	3787	3950	3787
q22	353	277	284	277
Total cold run time: 119310 ms
Total hot run time: 40765 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4361	4265	4206	4206
q2	385	256	260	256
q3	2996	2801	2667	2667
q4	1843	1622	1610	1610
q5	5263	5280	5298	5280
q6	216	128	127	127
q7	2140	1753	1765	1753
q8	3204	3330	3309	3309
q9	8279	8321	8297	8297
q10	3912	3690	3690	3690
q11	581	504	492	492
q12	763	578	615	578
q13	16479	3004	3003	3003
q14	282	253	272	253
q15	517	483	480	480
q16	473	421	426	421
q17	1802	1523	1492	1492
q18	7683	7511	7506	7506
q19	1660	1561	1554	1554
q20	1974	1807	1784	1784
q21	4856	4900	4681	4681
q22	574	507	478	478
Total cold run time: 70243 ms
Total hot run time: 53917 ms

doris-robot · 2024-05-24T08:24:55Z

TPC-DS: Total hot run time: 172652 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0dc1c1d8e6555f1ea99b0d74c5b400832a832fa9, data reload: false
query1	927	399	384	384
query2	6478	2412	2401	2401
query3	6659	204	210	204
query4	19987	17463	17422	17422
query5	4149	429	423	423
query6	251	162	167	162
query7	4578	308	300	300
query8	254	191	191	191
query9	8530	2424	2391	2391
query10	451	293	271	271
query11	11087	10028	10028	10028
query12	145	94	92	92
query13	1657	365	371	365
query14	10232	7646	7569	7569
query15	264	173	170	170
query16	8162	276	263	263
query17	1831	548	537	537
query18	2125	278	278	278
query19	283	163	167	163
query20	100	86	86	86
query21	197	130	131	130
query22	4169	4068	3895	3895
query23	33738	33110	33015	33015
query24	11759	2809	2752	2752
query25	629	352	359	352
query26	1744	177	155	155
query27	2929	323	326	323
query28	7085	2075	2078	2075
query29	1018	631	594	594
query30	286	171	175	171
query31	962	740	742	740
query32	84	51	54	51
query33	766	281	275	275
query34	1022	469	477	469
query35	743	600	609	600
query36	1081	941	925	925
query37	274	72	69	69
query38	2889	2758	2795	2758
query39	866	772	790	772
query40	274	129	127	127
query41	49	44	43	43
query42	107	97	99	97
query43	613	563	545	545
query44	1198	745	752	745
query45	180	171	165	165
query46	1093	735	719	719
query47	1832	1779	1745	1745
query48	380	296	302	296
query49	1196	414	391	391
query50	784	397	399	397
query51	6781	6804	6585	6585
query52	111	98	95	95
query53	360	289	289	289
query54	997	450	434	434
query55	73	73	74	73
query56	272	241	252	241
query57	1144	1016	1059	1016
query58	234	214	211	211
query59	3464	3337	3212	3212
query60	280	255	260	255
query61	97	117	93	93
query62	654	455	475	455
query63	315	285	280	280
query64	9787	2230	1702	1702
query65	3455	3076	3161	3076
query66	1389	340	326	326
query67	15147	14903	14714	14714
query68	4778	537	542	537
query69	462	268	269	268
query70	1166	1068	1143	1068
query71	419	269	269	269
query72	7260	5681	5268	5268
query73	720	321	325	321
query74	6106	5578	5575	5575
query75	3550	2672	2617	2617
query76	3037	984	962	962
query77	654	273	276	273
query78	10331	9780	10091	9780
query79	2144	519	533	519
query80	960	452	443	443
query81	536	245	240	240
query82	613	92	94	92
query83	287	232	171	171
query84	240	87	92	87
query85	1634	280	263	263
query86	503	316	279	279
query87	3259	3062	3120	3062
query88	4161	2352	2360	2352
query89	474	393	380	380
query90	2029	191	190	190
query91	125	97	97	97
query92	57	50	48	48
query93	2093	520	499	499
query94	1212	192	193	192
query95	411	314	317	314
query96	591	273	266	266
query97	3236	3001	3008	3001
query98	230	233	221	221
query99	1182	850	854	850
Total cold run time: 277117 ms
Total hot run time: 172652 ms

doris-robot · 2024-05-24T08:30:25Z

ClickBench: Total hot run time: 30.08 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0dc1c1d8e6555f1ea99b0d74c5b400832a832fa9, data reload: false
query1	0.04	0.03	0.04
query2	0.08	0.05	0.05
query3	0.23	0.04	0.05
query4	1.67	0.07	0.08
query5	0.50	0.50	0.50
query6	1.11	0.72	0.71
query7	0.02	0.02	0.01
query8	0.06	0.04	0.05
query9	0.52	0.50	0.49
query10	0.54	0.55	0.54
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.59	0.59	0.59
query14	0.77	0.78	0.80
query15	0.83	0.80	0.80
query16	0.35	0.35	0.35
query17	0.92	0.99	0.95
query18	0.20	0.23	0.24
query19	1.74	1.68	1.68
query20	0.01	0.01	0.02
query21	15.44	0.70	0.69
query22	3.74	8.33	1.56
query23	18.33	1.42	1.25
query24	2.04	0.20	0.21
query25	0.14	0.08	0.08
query26	0.27	0.16	0.17
query27	0.08	0.07	0.08
query28	13.32	1.02	0.99
query29	12.69	3.27	3.23
query30	0.25	0.06	0.06
query31	2.86	0.38	0.37
query32	3.29	0.47	0.46
query33	2.86	2.89	2.90
query34	17.12	4.41	4.44
query35	4.51	4.53	4.52
query36	0.64	0.46	0.46
query37	0.18	0.15	0.15
query38	0.16	0.15	0.14
query39	0.04	0.04	0.04
query40	0.17	0.14	0.15
query41	0.09	0.05	0.04
query42	0.06	0.04	0.04
query43	0.04	0.04	0.03
Total cold run time: 108.8 s
Total hot run time: 30.08 s

doris-robot · 2024-05-24T08:53:57Z

TeamCity be ut coverage result:
Function Coverage: 35.65% (9019/25296)
Line Coverage: 27.31% (74591/273086)
Region Coverage: 26.55% (38613/145452)
Branch Coverage: 23.41% (19697/84150)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0dc1c1d8e6555f1ea99b0d74c5b400832a832fa9_0dc1c1d8e6555f1ea99b0d74c5b400832a832fa9/report/index.html

yujun777 · 2024-05-27T01:56:02Z

run p0

dataroaring

httpstream should be handled
Test cases should include insert into values and what about if a fe is restated.

dataroaring · 2024-05-27T03:13:00Z

 }
 request.__set_request_id(ctx->id.to_thrift());
+ request.__set_backend_id(_exec_env->master_info()->backend_id);



httpstream shoud also be considered.

had fix httpstream

dataroaring · 2024-05-27T03:20:23Z

Please implement it in cloud mode.

github-actions · 2024-06-05T07:29:36Z

clang-tidy review says "All clean, LGTM! 👍"

yujun777 · 2024-06-06T07:26:56Z

run buildall

doris-robot · 2024-06-06T08:33:27Z

TPC-H: Total hot run time: 41105 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7adb998690d40e7ae69ef3ad85e563b0300dfd7f, data reload: false
------ Round 1 ----------------------------------
q1	17677	4402	4282	4282
q2	2035	205	204	204
q3	10492	1313	1248	1248
q4	10210	832	756	756
q5	7494	2766	2728	2728
q6	223	137	143	137
q7	996	622	622	622
q8	9220	2143	2136	2136
q9	9342	6752	6691	6691
q10	9703	3901	3890	3890
q11	446	244	253	244
q12	401	230	240	230
q13	17396	3206	3301	3206
q14	268	212	225	212
q15	530	471	482	471
q16	506	400	389	389
q17	1008	622	698	622
q18	8583	7921	7730	7730
q19	8746	1446	1358	1358
q20	655	344	319	319
q21	5210	4018	3295	3295
q22	395	335	341	335
Total cold run time: 121536 ms
Total hot run time: 41105 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4583	4431	4466	4431
q2	391	274	267	267
q3	3164	2944	2951	2944
q4	1856	1566	1662	1566
q5	5507	5519	5517	5517
q6	220	125	129	125
q7	2188	1833	1777	1777
q8	3249	3415	3415	3415
q9	8685	8701	8669	8669
q10	4046	3811	3717	3717
q11	588	491	491	491
q12	771	627	608	608
q13	17036	3170	3097	3097
q14	312	303	284	284
q15	520	479	479	479
q16	487	427	418	418
q17	1839	1483	1492	1483
q18	8273	7535	7294	7294
q19	1767	1561	1521	1521
q20	3054	1774	1787	1774
q21	9432	4623	4749	4623
q22	625	548	577	548
Total cold run time: 78593 ms
Total hot run time: 55048 ms

doris-robot · 2024-06-06T08:44:24Z

TPC-DS: Total hot run time: 172354 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7adb998690d40e7ae69ef3ad85e563b0300dfd7f, data reload: false
query1	921	387	371	371
query2	6473	2412	2456	2412
query3	6639	217	216	216
query4	19395	17425	17226	17226
query5	4137	457	458	457
query6	241	151	155	151
query7	4587	311	294	294
query8	330	293	284	284
query9	8662	2414	2403	2403
query10	469	314	282	282
query11	11706	10020	10065	10020
query12	136	85	87	85
query13	1665	371	357	357
query14	10131	7577	7586	7577
query15	237	184	193	184
query16	7810	265	265	265
query17	1509	522	527	522
query18	1897	264	264	264
query19	199	155	147	147
query20	90	83	80	80
query21	199	129	123	123
query22	4246	3955	4041	3955
query23	33638	32946	33295	32946
query24	11032	2892	2789	2789
query25	622	349	363	349
query26	1557	156	152	152
query27	2980	319	332	319
query28	7500	2060	2063	2060
query29	1001	610	597	597
query30	256	156	153	153
query31	949	745	718	718
query32	93	52	55	52
query33	775	294	321	294
query34	893	470	459	459
query35	749	607	615	607
query36	1090	922	912	912
query37	152	68	70	68
query38	2828	2752	2733	2733
query39	860	792	788	788
query40	229	122	125	122
query41	54	53	52	52
query42	115	99	96	96
query43	584	559	523	523
query44	1212	720	737	720
query45	198	169	168	168
query46	1070	717	715	715
query47	1816	1764	1799	1764
query48	382	295	299	295
query49	1043	399	401	399
query50	789	378	390	378
query51	6796	6650	6663	6650
query52	101	94	102	94
query53	360	295	325	295
query54	912	443	434	434
query55	75	75	74	74
query56	276	261	262	261
query57	1175	1036	1091	1036
query58	253	272	241	241
query59	3575	3263	3385	3263
query60	293	295	273	273
query61	106	87	86	86
query62	639	455	452	452
query63	323	290	293	290
query64	9862	2239	1686	1686
query65	3171	3099	3115	3099
query66	1341	331	333	331
query67	15269	14785	14923	14785
query68	4525	558	557	557
query69	458	308	294	294
query70	1177	1058	1036	1036
query71	390	270	261	261
query72	7023	5074	5573	5074
query73	736	328	326	326
query74	5928	5779	5495	5495
query75	3407	2700	2635	2635
query76	2304	931	912	912
query77	457	300	302	300
query78	10214	9931	9640	9640
query79	2473	520	516	516
query80	1087	459	445	445
query81	542	221	218	218
query82	742	104	104	104
query83	233	171	174	171
query84	255	91	84	84
query85	1904	293	266	266
query86	517	330	293	293
query87	3288	3096	3061	3061
query88	3792	2409	2358	2358
query89	478	401	379	379
query90	1880	190	190	190
query91	129	96	96	96
query92	64	51	51	51
query93	2050	517	506	506
query94	1251	194	184	184
query95	407	313	316	313
query96	581	271	274	271
query97	3160	3051	3035	3035
query98	226	206	201	201
query99	1136	829	830	829
Total cold run time: 273487 ms
Total hot run time: 172354 ms

doris-robot · 2024-06-06T08:49:55Z

ClickBench: Total hot run time: 30.4 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7adb998690d40e7ae69ef3ad85e563b0300dfd7f, data reload: false
query1	0.04	0.04	0.04
query2	0.09	0.04	0.05
query3	0.22	0.05	0.06
query4	1.69	0.07	0.08
query5	0.48	0.47	0.49
query6	1.12	0.72	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.53	0.48	0.47
query10	0.54	0.56	0.54
query11	0.15	0.11	0.12
query12	0.14	0.12	0.12
query13	0.59	0.60	0.61
query14	0.78	0.81	0.76
query15	0.84	0.81	0.81
query16	0.37	0.37	0.37
query17	1.01	1.01	1.02
query18	0.24	0.24	0.26
query19	1.80	1.74	1.68
query20	0.01	0.01	0.01
query21	15.54	0.68	0.66
query22	3.61	7.72	1.81
query23	18.31	1.31	1.22
query24	2.14	0.22	0.23
query25	0.16	0.09	0.08
query26	0.26	0.17	0.17
query27	0.08	0.08	0.08
query28	13.20	1.02	0.99
query29	13.39	3.36	3.31
query30	0.24	0.07	0.05
query31	2.86	0.38	0.39
query32	3.29	0.47	0.47
query33	2.92	2.89	2.95
query34	17.15	4.39	4.39
query35	4.46	4.49	4.63
query36	0.68	0.47	0.46
query37	0.18	0.14	0.15
query38	0.15	0.14	0.14
query39	0.05	0.03	0.04
query40	0.18	0.17	0.14
query41	0.08	0.05	0.04
query42	0.05	0.05	0.05
query43	0.04	0.04	0.03
Total cold run time: 109.73 s
Total hot run time: 30.4 s

doris-robot · 2024-06-06T08:54:39Z

TeamCity be ut coverage result:
Function Coverage: 36.36% (8987/24714)
Line Coverage: 27.90% (73393/263073)
Region Coverage: 27.32% (37995/139072)
Branch Coverage: 23.91% (19274/80604)
Coverage Report: http://coverage.selectdb-in.cc/coverage/7adb998690d40e7ae69ef3ad85e563b0300dfd7f_7adb998690d40e7ae69ef3ad85e563b0300dfd7f/report/index.html

github-actions · 2024-06-07T04:00:10Z

clang-tidy review says "All clean, LGTM! 👍"

yujun777 · 2024-06-11T06:02:38Z

run buildall

doris-robot · 2024-06-11T07:15:53Z

TPC-H: Total hot run time: 40701 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c5741a5f0a69e380f0c9bcfa0679160c2e318ea6, data reload: false
------ Round 1 ----------------------------------
q1	17713	4549	4301	4301
q2	2039	190	195	190
q3	10524	1226	1151	1151
q4	10225	894	793	793
q5	7455	2766	2706	2706
q6	224	139	139	139
q7	957	629	604	604
q8	9219	2106	2102	2102
q9	8979	6521	6447	6447
q10	9000	3744	3801	3744
q11	473	236	246	236
q12	407	226	226	226
q13	17851	2975	3015	2975
q14	259	234	221	221
q15	523	491	493	491
q16	536	386	372	372
q17	989	736	721	721
q18	8154	7469	7363	7363
q19	5503	1526	1396	1396
q20	655	328	319	319
q21	4933	3861	3954	3861
q22	410	343	343	343
Total cold run time: 117028 ms
Total hot run time: 40701 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4349	4217	4204	4204
q2	381	285	273	273
q3	3059	2933	2914	2914
q4	1959	1682	1716	1682
q5	5573	5575	5547	5547
q6	219	128	134	128
q7	2235	1886	1878	1878
q8	3259	3448	3428	3428
q9	8803	8805	8863	8805
q10	4178	3825	3812	3812
q11	590	493	486	486
q12	803	630	647	630
q13	16024	3148	3166	3148
q14	301	277	280	277
q15	516	489	476	476
q16	491	430	436	430
q17	1843	1523	1501	1501
q18	8170	8064	7850	7850
q19	2611	1629	1639	1629
q20	2108	1854	1866	1854
q21	5167	4867	4851	4851
q22	618	581	585	581
Total cold run time: 73257 ms
Total hot run time: 56384 ms

doris-robot · 2024-06-11T07:26:53Z

TPC-DS: Total hot run time: 173546 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c5741a5f0a69e380f0c9bcfa0679160c2e318ea6, data reload: false
query1	940	384	383	383
query2	6460	2482	2577	2482
query3	6634	209	209	209
query4	19103	17408	17046	17046
query5	3604	474	453	453
query6	240	160	166	160
query7	4580	306	303	303
query8	341	283	286	283
query9	8476	2384	2337	2337
query10	551	303	273	273
query11	10501	10116	9880	9880
query12	119	90	95	90
query13	1632	371	366	366
query14	10111	7079	7865	7079
query15	283	193	202	193
query16	7435	255	263	255
query17	1920	553	516	516
query18	1954	271	267	267
query19	197	177	146	146
query20	91	81	82	81
query21	207	127	126	126
query22	4356	4028	3973	3973
query23	34072	33528	33623	33528
query24	11222	3002	2898	2898
query25	605	361	420	361
query26	1267	157	152	152
query27	2819	331	326	326
query28	7227	2116	2111	2111
query29	871	632	633	632
query30	250	151	154	151
query31	940	767	752	752
query32	102	55	57	55
query33	750	279	275	275
query34	1089	489	498	489
query35	718	660	623	623
query36	1128	983	975	975
query37	166	70	72	70
query38	2907	2857	2829	2829
query39	914	850	827	827
query40	224	125	126	125
query41	60	53	58	53
query42	119	104	110	104
query43	606	569	542	542
query44	1263	725	734	725
query45	195	161	166	161
query46	1077	716	722	716
query47	1831	1745	1767	1745
query48	359	293	291	291
query49	853	400	400	400
query50	754	391	393	391
query51	6827	6764	6584	6584
query52	107	89	98	89
query53	352	291	284	284
query54	844	460	446	446
query55	75	72	72	72
query56	268	254	294	254
query57	1099	1052	1066	1052
query58	251	250	272	250
query59	3488	3228	3198	3198
query60	293	266	276	266
query61	94	85	99	85
query62	591	448	450	448
query63	317	284	290	284
query64	8928	2205	1809	1809
query65	3203	3061	3094	3061
query66	756	330	337	330
query67	15461	14880	14989	14880
query68	4454	544	544	544
query69	458	326	320	320
query70	1163	1059	1138	1059
query71	401	279	275	275
query72	7150	5574	5746	5574
query73	758	326	326	326
query74	5862	5499	5465	5465
query75	3417	2651	2681	2651
query76	2281	884	950	884
query77	426	300	298	298
query78	10463	9952	9881	9881
query79	3551	524	538	524
query80	1325	495	485	485
query81	584	231	225	225
query82	783	110	102	102
query83	210	178	178	178
query84	278	87	96	87
query85	1386	326	316	316
query86	474	316	322	316
query87	3219	3060	3044	3044
query88	4642	2389	2368	2368
query89	490	390	392	390
query90	1817	194	189	189
query91	139	252	98	98
query92	59	48	50	48
query93	4390	519	502	502
query94	1136	229	183	183
query95	395	315	300	300
query96	612	268	264	264
query97	3186	3017	3061	3017
query98	219	194	195	194
query99	1184	841	839	839
Total cold run time: 273989 ms
Total hot run time: 173546 ms

doris-robot · 2024-06-11T07:32:24Z

ClickBench: Total hot run time: 30.97 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c5741a5f0a69e380f0c9bcfa0679160c2e318ea6, data reload: false
query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.24	0.05	0.05
query4	1.67	0.08	0.07
query5	0.50	0.47	0.47
query6	1.13	0.73	0.73
query7	0.01	0.02	0.02
query8	0.06	0.05	0.05
query9	0.53	0.50	0.50
query10	0.55	0.56	0.54
query11	0.15	0.12	0.12
query12	0.14	0.12	0.13
query13	0.59	0.59	0.61
query14	0.78	0.80	0.76
query15	0.84	0.82	0.83
query16	0.37	0.35	0.38
query17	0.96	1.03	1.04
query18	0.22	0.23	0.26
query19	1.80	1.69	1.86
query20	0.02	0.01	0.01
query21	15.41	0.65	0.64
query22	3.77	7.38	2.27
query23	18.28	1.42	1.26
query24	2.07	0.22	0.21
query25	0.17	0.09	0.10
query26	0.27	0.17	0.18
query27	0.08	0.09	0.08
query28	13.27	1.02	1.02
query29	12.58	3.31	3.32
query30	0.25	0.06	0.06
query31	2.85	0.39	0.38
query32	3.28	0.48	0.47
query33	2.90	2.94	2.92
query34	16.97	4.36	4.40
query35	4.46	4.47	4.46
query36	0.66	0.48	0.46
query37	0.18	0.16	0.16
query38	0.15	0.14	0.14
query39	0.05	0.03	0.03
query40	0.16	0.14	0.16
query41	0.10	0.04	0.04
query42	0.05	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 108.68 s
Total hot run time: 30.97 s

doris-robot · 2024-06-11T07:38:14Z

TeamCity be ut coverage result:
Function Coverage: 36.45% (8977/24631)
Line Coverage: 27.98% (73378/262248)
Region Coverage: 27.37% (38016/138912)
Branch Coverage: 24.00% (19309/80466)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c5741a5f0a69e380f0c9bcfa0679160c2e318ea6_c5741a5f0a69e380f0c9bcfa0679160c2e318ea6/report/index.html

dataroaring · 2024-06-13T07:13:51Z

 @SerializedName(value = "ip")
 public String ip;
+ // frontend/backend start time
+ @SerializedName(value = "startTime")


short name is preferred, like st.

change to st

github-actions · 2024-06-13T10:18:01Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-06-13T10:19:20Z

clang-tidy review says "All clean, LGTM! 👍"

yujun777 · 2024-06-14T06:15:46Z

run buildall

doris-robot · 2024-06-14T07:31:23Z

TPC-H: Total hot run time: 40520 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit daf6f984b975220fb6659693ce86a269b23d5a6d, data reload: false
------ Round 1 ----------------------------------
q1	17628	4434	4235	4235
q2	2020	195	193	193
q3	10445	1189	1103	1103
q4	10196	783	753	753
q5	7495	2682	2630	2630
q6	220	139	137	137
q7	959	609	591	591
q8	9219	2092	2085	2085
q9	8974	6488	6473	6473
q10	8931	3720	3671	3671
q11	457	240	241	240
q12	551	233	245	233
q13	18729	2987	2975	2975
q14	282	205	224	205
q15	521	474	495	474
q16	512	381	378	378
q17	971	676	682	676
q18	7991	7378	7374	7374
q19	6400	1566	1518	1518
q20	649	317	318	317
q21	4926	3920	3966	3920
q22	394	346	339	339
Total cold run time: 118470 ms
Total hot run time: 40520 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4356	4252	4233	4233
q2	363	269	273	269
q3	3105	2911	2921	2911
q4	1954	1718	1769	1718
q5	5498	5587	5468	5468
q6	225	131	129	129
q7	2238	1806	1861	1806
q8	3296	3462	3431	3431
q9	8788	8790	8808	8790
q10	4131	3780	3796	3780
q11	616	482	501	482
q12	814	632	632	632
q13	16242	3199	3177	3177
q14	298	286	267	267
q15	542	483	510	483
q16	487	443	432	432
q17	1832	1513	1608	1513
q18	8054	7966	7901	7901
q19	1803	1578	1756	1578
q20	3092	1891	1838	1838
q21	5160	4856	4702	4702
q22	626	550	533	533
Total cold run time: 73520 ms
Total hot run time: 56073 ms

doris-robot · 2024-06-14T07:42:19Z

TPC-DS: Total hot run time: 174151 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit daf6f984b975220fb6659693ce86a269b23d5a6d, data reload: false
query1	935	388	384	384
query2	6453	2434	2544	2434
query3	6643	216	207	207
query4	21021	17288	17302	17288
query5	3639	500	461	461
query6	238	159	161	159
query7	4577	303	304	303
query8	337	308	286	286
query9	8430	2402	2400	2400
query10	568	319	275	275
query11	10604	10052	10019	10019
query12	117	92	92	92
query13	1634	365	365	365
query14	9763	6932	7537	6932
query15	244	183	189	183
query16	7554	263	265	263
query17	1483	532	511	511
query18	1887	270	277	270
query19	210	177	150	150
query20	92	83	84	83
query21	211	133	127	127
query22	4206	4100	4006	4006
query23	33888	33732	33627	33627
query24	10925	2917	2860	2860
query25	597	384	361	361
query26	731	151	159	151
query27	2352	319	325	319
query28	5877	2158	2119	2119
query29	890	655	653	653
query30	260	157	157	157
query31	955	773	774	773
query32	94	60	57	57
query33	643	285	279	279
query34	894	477	494	477
query35	734	639	618	618
query36	1117	983	985	983
query37	155	77	70	70
query38	3033	2846	2854	2846
query39	907	826	808	808
query40	240	152	133	133
query41	57	56	54	54
query42	113	106	105	105
query43	602	567	555	555
query44	1109	735	731	731
query45	201	170	165	165
query46	1066	695	701	695
query47	1846	1782	1741	1741
query48	380	297	293	293
query49	845	415	425	415
query50	770	398	404	398
query51	6927	6623	6732	6623
query52	101	94	96	94
query53	366	295	290	290
query54	850	469	457	457
query55	75	75	79	75
query56	301	282	280	280
query57	1135	1084	1017	1017
query58	265	264	256	256
query59	3466	3269	3272	3269
query60	308	291	291	291
query61	113	159	87	87
query62	611	438	447	438
query63	315	282	284	282
query64	8543	2248	1810	1810
query65	3177	3127	3093	3093
query66	771	339	324	324
query67	15504	15138	14935	14935
query68	4562	543	545	543
query69	571	374	347	347
query70	1141	1132	1135	1132
query71	409	273	278	273
query72	7759	5441	5517	5441
query73	748	323	323	323
query74	5788	5585	5463	5463
query75	3854	2647	2676	2647
query76	2380	910	894	894
query77	694	301	311	301
query78	10336	10051	9811	9811
query79	1936	525	517	517
query80	2662	474	475	474
query81	559	222	222	222
query82	1091	110	103	103
query83	326	169	226	169
query84	265	89	94	89
query85	1141	285	282	282
query86	458	341	312	312
query87	3316	3089	3093	3089
query88	2958	2363	2349	2349
query89	483	394	404	394
query90	1736	188	200	188
query91	130	103	102	102
query92	63	49	52	49
query93	2066	507	507	507
query94	1192	192	190	190
query95	414	315	322	315
query96	600	264	265	264
query97	3265	3059	3060	3059
query98	236	204	199	199
query99	1232	841	857	841
Total cold run time: 269326 ms
Total hot run time: 174151 ms

doris-robot · 2024-06-14T07:47:50Z

ClickBench: Total hot run time: 30.55 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit daf6f984b975220fb6659693ce86a269b23d5a6d, data reload: false
query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.05	0.06
query4	1.67	0.08	0.07
query5	0.50	0.48	0.49
query6	1.12	0.72	0.72
query7	0.02	0.01	0.01
query8	0.06	0.04	0.05
query9	0.54	0.50	0.46
query10	0.55	0.56	0.54
query11	0.15	0.11	0.12
query12	0.14	0.12	0.12
query13	0.60	0.59	0.60
query14	0.77	0.79	0.78
query15	0.83	0.81	0.82
query16	0.36	0.37	0.37
query17	1.03	0.97	1.03
query18	0.20	0.27	0.23
query19	1.82	1.70	1.73
query20	0.02	0.01	0.01
query21	15.43	0.66	0.66
query22	3.59	8.43	1.78
query23	18.25	1.27	1.30
query24	2.15	0.23	0.23
query25	0.15	0.10	0.09
query26	0.27	0.18	0.18
query27	0.09	0.08	0.08
query28	13.20	1.02	1.00
query29	12.63	3.29	3.27
query30	0.26	0.07	0.06
query31	2.85	0.39	0.40
query32	3.26	0.47	0.48
query33	2.84	2.91	2.92
query34	16.85	4.44	4.46
query35	4.49	4.54	4.55
query36	0.66	0.46	0.47
query37	0.18	0.15	0.15
query38	0.15	0.14	0.14
query39	0.05	0.04	0.03
query40	0.17	0.14	0.15
query41	0.09	0.05	0.04
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 108.43 s
Total hot run time: 30.55 s

doris-robot · 2024-06-14T07:52:58Z

TeamCity be ut coverage result:
Function Coverage: 36.44% (8986/24661)
Line Coverage: 28.02% (73665/262914)
Region Coverage: 27.49% (38270/139201)
Branch Coverage: 24.19% (19513/80658)
Coverage Report: http://coverage.selectdb-in.cc/coverage/daf6f984b975220fb6659693ce86a269b23d5a6d_daf6f984b975220fb6659693ce86a269b23d5a6d/report/index.html

dataroaring

LGTM

github-actions · 2024-06-14T11:25:35Z

PR approved by at least one committer and no changes requested.

deardeng

LGTM

BUG: fe will abort coordidator BE's txn when be shutdown exceeds 5min. But if BE restart within 5min, then this BE's txns will not abort until timeout. FIX: every txn will record BE's id and its start time. When fe found be's startTime change, it will abort the BE's old txns.

cherry pick from #35342

…#35342) (#39317) cherry-pick: #22781, #28662, #35342 --------- Co-authored-by: HHoflittlefish777 <77738092+HHoflittlefish777@users.noreply.github.com>

…pache#28662, apache#35342) (apache#39317) cherry-pick: apache#22781, apache#28662, apache#35342 --------- Co-authored-by: HHoflittlefish777 <77738092+HHoflittlefish777@users.noreply.github.com>

BUG: fe will abort coordidator BE's txn when be shutdown exceeds 5min. But if BE restart within 5min, then this BE's txns will not abort until timeout. FIX: every txn will record BE's id and its start time. When fe found be's startTime change, it will abort the BE's old txns.

github-actionsBot added the meta-change label May 24, 2024

924060929 previously approved these changes May 24, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label May 24, 2024

github-actionsBot added the reviewed label May 24, 2024

dataroaring requested changes May 27, 2024
View reviewed changes

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label May 27, 2024

yujun777 force-pushed the coordidator-be-restart-abort-txn branch from 0dc1c1d to 7adb998Compare June 5, 2024 07:23

yujun777 dismissed 924060929’s stale review via c5741a5June 7, 2024 03:54

dataroaring reviewed Jun 13, 2024
View reviewed changes

yujun777 added 2 commits June 13, 2024 18:13

coordidator be restart abort txn
a979767

update
daf6f98

yujun777 force-pushed the coordidator-be-restart-abort-txn branch from 03abb17 to daf6f98Compare June 13, 2024 10:13

dataroaring approved these changes Jun 14, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jun 14, 2024

deardeng approved these changes Jun 14, 2024
View reviewed changes

dataroaring merged commit 815c7e6 into apache:masterJun 14, 2024

dataroaring added the dev/2.1.x label Jun 14, 2024

yujun777 mentioned this pull request Jun 18, 2024
[fix](txn) Fix coordidator be restart not abort txn #35342 #36437
Merged

dataroaring pushed a commit that referenced this pull request Jun 25, 2024
[fix](txn) Fix coordidator be restart not abort txn#35342(#36437)
785a1f4
cherry pick from #35342

dataroaring added dev/2.1.5-merged and removed dev/2.1.x labels Jun 25, 2024

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

yujun777 mentioned this pull request Aug 13, 2024
[branch-2.0](txn) be dead exceeds 5min abort its txns (#22781, #28662, #35342) #39317
Merged

dataroaring added the dev/2.0.15-merged label Aug 14, 2024

Uh oh!

Conversation

yujun777 commented May 24, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

doris-robot commented May 24, 2024

Uh oh!

github-actionsBot commented May 24, 2024

Uh oh!

yujun777 commented May 24, 2024

Uh oh!

github-actionsBot commented May 24, 2024

Uh oh!

github-actionsBot commented May 24, 2024

Uh oh!

doris-robot commented May 24, 2024

Uh oh!

doris-robot commented May 24, 2024

Uh oh!

doris-robot commented May 24, 2024

Uh oh!

doris-robot commented May 24, 2024

Uh oh!

yujun777 commented May 27, 2024

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

dataroaringMay 27, 2024

Choose a reason for hiding this comment

Uh oh!

yujun777Jun 14, 2024

Choose a reason for hiding this comment

Uh oh!

dataroaring commented May 27, 2024

Uh oh!

github-actionsBot commented Jun 5, 2024

Uh oh!

yujun777 commented Jun 6, 2024

Uh oh!

doris-robot commented Jun 6, 2024

Uh oh!

doris-robot commented Jun 6, 2024

Uh oh!

doris-robot commented Jun 6, 2024

Uh oh!

doris-robot commented Jun 6, 2024

Uh oh!

github-actionsBot commented Jun 7, 2024

Uh oh!

yujun777 commented Jun 11, 2024

Uh oh!

doris-robot commented Jun 11, 2024

Uh oh!

doris-robot commented Jun 11, 2024

Uh oh!

doris-robot commented Jun 11, 2024

Uh oh!

doris-robot commented Jun 11, 2024

Uh oh!

dataroaringJun 13, 2024

Choose a reason for hiding this comment

Uh oh!

yujun777Jun 14, 2024

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jun 13, 2024

Uh oh!

github-actionsBot commented Jun 13, 2024

Uh oh!

yujun777 commented Jun 14, 2024

Uh oh!

doris-robot commented Jun 14, 2024

Uh oh!

doris-robot commented Jun 14, 2024

Uh oh!

doris-robot commented Jun 14, 2024

Uh oh!

yujun777 commented May 24, 2024•
edited
Loading