[fix](pipelinex) fix fragment instance progress reports by kaijchen · Pull Request #40325 · apache/doris

kaijchen · 2024-09-03T09:30:17Z

Proposed changes

Fix progress in show load not 100% when load finished.

PipelineX has deprecated fragmentInstanceId in TReportExecStatusParams.
The finishedScanNums should be reported with distinct fragmentInstanceId to prevent being overwritten.

Before

| 38024 | lineitem_dup_1_replication_s3_2 | FINISHED | 30.00% (3/10) | BROKER | ...

After

| 39025 | lineitem_dup_1_replication_s3_3 | FINISHED | 100.00% (10/10) | BROKER | ...

doris-robot · 2024-09-03T09:30:22Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

kaijchen · 2024-09-03T09:33:03Z

run buildall

dataroaring

LGTM

github-actions · 2024-09-03T09:41:16Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-09-03T09:41:18Z

PR approved by anyone and no changes requested.

doris-robot · 2024-09-03T09:53:02Z

TPC-H: Total hot run time: 38440 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 745d41dee6ba01cae6d00586483a2a9474b4113b, data reload: false
------ Round 1 ----------------------------------
q1	18209	4504	4418	4418
q2	2024	182	174	174
q3	10490	1176	1126	1126
q4	10138	788	649	649
q5	7709	2866	2804	2804
q6	237	139	139	139
q7	982	619	615	615
q8	9307	2069	2082	2069
q9	7178	6533	6561	6533
q10	7001	2242	2272	2242
q11	463	246	248	246
q12	417	227	219	219
q13	17758	3116	3095	3095
q14	276	236	241	236
q15	525	474	495	474
q16	611	514	507	507
q17	980	739	776	739
q18	7435	6882	7067	6882
q19	1396	1117	1067	1067
q20	692	317	318	317
q21	4102	3193	2907	2907
q22	1081	996	982	982
Total cold run time: 109011 ms
Total hot run time: 38440 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4480	4324	4302	4302
q2	390	290	271	271
q3	2929	2745	2691	2691
q4	1963	1698	1665	1665
q5	5449	5408	5432	5408
q6	223	132	132	132
q7	2123	1795	1769	1769
q8	3244	3376	3414	3376
q9	8453	8485	8512	8485
q10	3488	3203	3214	3203
q11	597	521	491	491
q12	792	618	602	602
q13	13339	3106	3091	3091
q14	308	277	275	275
q15	530	498	476	476
q16	623	574	570	570
q17	1810	1512	1516	1512
q18	7946	7668	7580	7580
q19	1675	1556	1460	1460
q20	2103	1830	1848	1830
q21	5599	5438	5368	5368
q22	1114	1039	992	992
Total cold run time: 69178 ms
Total hot run time: 55549 ms

doris-robot · 2024-09-03T10:04:43Z

TPC-DS: Total hot run time: 187205 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 745d41dee6ba01cae6d00586483a2a9474b4113b, data reload: false
query1	916	375	364	364
query2	6462	2036	1936	1936
query3	6643	206	213	206
query4	34057	23087	23214	23087
query5	4189	530	481	481
query6	255	169	185	169
query7	4580	293	282	282
query8	249	206	199	199
query9	8575	2425	2433	2425
query10	436	253	258	253
query11	16759	15116	15117	15116
query12	159	97	100	97
query13	1641	368	364	364
query14	9673	6822	6402	6402
query15	263	171	177	171
query16	8024	459	455	455
query17	1625	573	555	555
query18	2007	298	298	298
query19	197	151	149	149
query20	115	111	109	109
query21	208	107	103	103
query22	4523	4170	4389	4170
query23	34127	33679	33527	33527
query24	10965	2928	2850	2850
query25	619	404	380	380
query26	1234	154	151	151
query27	2587	275	277	275
query28	7308	2030	2013	2013
query29	808	416	403	403
query30	296	170	154	154
query31	977	762	801	762
query32	98	55	61	55
query33	728	286	284	284
query34	956	481	483	481
query35	862	784	718	718
query36	1055	913	938	913
query37	172	90	97	90
query38	4046	3882	3890	3882
query39	1442	1407	1383	1383
query40	204	114	111	111
query41	48	47	46	46
query42	114	91	95	91
query43	519	488	482	482
query44	1188	731	719	719
query45	200	171	170	170
query46	1100	718	727	718
query47	1873	1792	1794	1792
query48	373	294	289	289
query49	1096	417	439	417
query50	821	397	395	395
query51	7027	6871	6971	6871
query52	104	88	84	84
query53	252	176	181	176
query54	837	458	477	458
query55	74	76	74	74
query56	289	247	248	247
query57	1190	1079	1047	1047
query58	236	228	239	228
query59	3078	2937	2919	2919
query60	293	261	262	261
query61	100	100	102	100
query62	832	625	662	625
query63	223	179	179	179
query64	4104	751	713	713
query65	3259	3127	3132	3127
query66	1445	331	337	331
query67	15554	15449	15263	15263
query68	4507	566	536	536
query69	415	284	272	272
query70	1194	1138	1040	1040
query71	320	275	268	268
query72	6724	4085	4045	4045
query73	756	323	334	323
query74	9132	8878	8831	8831
query75	3411	2736	2685	2685
query76	2480	1049	1043	1043
query77	509	313	323	313
query78	9942	9021	9090	9021
query79	1013	533	534	533
query80	738	505	497	497
query81	451	232	236	232
query82	244	139	142	139
query83	170	154	150	150
query84	220	80	73	73
query85	696	278	278	278
query86	303	301	283	283
query87	4433	4367	4236	4236
query88	2934	2302	2324	2302
query89	381	293	283	283
query90	1950	187	187	187
query91	123	120	102	102
query92	59	48	49	48
query93	1043	524	524	524
query94	790	346	289	289
query95	352	249	247	247
query96	589	269	263	263
query97	3235	3047	3056	3047
query98	219	209	194	194
query99	1441	1283	1276	1276
Total cold run time: 287043 ms
Total hot run time: 187205 ms

doris-robot · 2024-09-03T10:10:15Z

ClickBench: Total hot run time: 32.34 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 745d41dee6ba01cae6d00586483a2a9474b4113b, data reload: false
query1	0.05	0.04	0.04
query2	0.07	0.04	0.04
query3	0.23	0.05	0.04
query4	1.68	0.07	0.06
query5	0.49	0.50	0.49
query6	1.14	0.73	0.73
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.55	0.48	0.50
query10	0.55	0.55	0.56
query11	0.15	0.12	0.11
query12	0.15	0.13	0.12
query13	0.61	0.59	0.59
query14	2.06	2.03	2.12
query15	0.89	0.83	0.81
query16	0.38	0.38	0.39
query17	0.98	1.04	1.05
query18	0.21	0.20	0.21
query19	1.88	1.87	1.74
query20	0.01	0.01	0.01
query21	15.40	0.66	0.65
query22	4.48	6.42	2.26
query23	18.32	1.39	1.28
query24	2.09	0.23	0.23
query25	0.15	0.09	0.07
query26	0.26	0.18	0.18
query27	0.07	0.08	0.07
query28	13.24	1.01	1.01
query29	12.64	3.29	3.34
query30	0.25	0.05	0.06
query31	2.86	0.39	0.39
query32	3.27	0.49	0.49
query33	2.98	3.03	2.98
query34	17.19	4.39	4.42
query35	4.46	4.41	4.46
query36	0.64	0.48	0.49
query37	0.18	0.15	0.15
query38	0.16	0.15	0.15
query39	0.05	0.04	0.03
query40	0.16	0.12	0.12
query41	0.10	0.05	0.05
query42	0.06	0.04	0.04
query43	0.04	0.05	0.05
Total cold run time: 111.2 s
Total hot run time: 32.34 s

github-actions · 2024-09-04T02:05:23Z

clang-tidy review says "All clean, LGTM! 👍"

kaijchen · 2024-09-04T02:45:47Z

run buildall

doris-robot · 2024-09-04T04:36:10Z

TPC-H: Total hot run time: 38155 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7b1cae23c02feafeae3acc0f8c781522235618c0, data reload: false
------ Round 1 ----------------------------------
q1	18165	5419	4317	4317
q2	2033	187	172	172
q3	11679	1049	1069	1049
q4	10253	715	744	715
q5	7780	2882	2807	2807
q6	234	139	140	139
q7	948	612	616	612
q8	9320	2080	2078	2078
q9	7266	6548	6570	6548
q10	6990	2235	2259	2235
q11	484	242	243	242
q12	397	225	225	225
q13	17843	3038	3109	3038
q14	280	234	236	234
q15	533	482	500	482
q16	569	501	513	501
q17	997	761	797	761
q18	7399	7004	6865	6865
q19	1382	1069	999	999
q20	712	332	327	327
q21	3903	2793	3214	2793
q22	1119	1016	1040	1016
Total cold run time: 110286 ms
Total hot run time: 38155 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4366	4345	4282	4282
q2	377	273	277	273
q3	2884	2637	2638	2637
q4	1912	1671	1665	1665
q5	5695	5744	5796	5744
q6	225	134	137	134
q7	2202	1864	1839	1839
q8	3312	3466	3557	3466
q9	8878	8838	8810	8810
q10	3626	3380	3404	3380
q11	597	495	521	495
q12	819	666	652	652
q13	16375	3206	3300	3206
q14	313	305	277	277
q15	541	478	500	478
q16	622	603	572	572
q17	1861	1549	1553	1549
q18	8137	8035	7979	7979
q19	1727	1669	1535	1535
q20	2166	1926	1943	1926
q21	5628	5460	5437	5437
q22	1155	1040	1082	1040
Total cold run time: 73418 ms
Total hot run time: 57376 ms

doris-robot · 2024-09-04T04:48:23Z

TPC-DS: Total hot run time: 192550 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7b1cae23c02feafeae3acc0f8c781522235618c0, data reload: false
query1	1278	904	866	866
query2	6338	1938	1835	1835
query3	10566	4030	3965	3965
query4	59456	27764	23228	23228
query5	5288	498	509	498
query6	416	153	160	153
query7	5786	287	285	285
query8	289	208	217	208
query9	9043	2500	2484	2484
query10	487	283	261	261
query11	17436	14969	15391	14969
query12	146	99	102	99
query13	1561	393	384	384
query14	10608	7355	7416	7355
query15	222	176	176	176
query16	7473	419	461	419
query17	1121	567	557	557
query18	2023	306	305	305
query19	286	160	161	160
query20	126	112	111	111
query21	209	114	106	106
query22	4607	4671	4693	4671
query23	34522	33396	33243	33243
query24	5926	2857	2835	2835
query25	550	398	376	376
query26	669	174	150	150
query27	1753	277	275	275
query28	3947	2057	2052	2052
query29	681	406	405	405
query30	239	154	157	154
query31	922	746	772	746
query32	78	49	51	49
query33	476	282	280	280
query34	873	470	475	470
query35	824	738	728	728
query36	1032	927	929	927
query37	155	88	86	86
query38	4056	3999	3897	3897
query39	1451	1411	1419	1411
query40	213	115	117	115
query41	48	48	47	47
query42	118	98	97	97
query43	513	464	475	464
query44	1104	739	736	736
query45	204	171	170	170
query46	1110	731	738	731
query47	1908	1844	1808	1808
query48	375	294	338	294
query49	751	429	430	429
query50	817	421	417	417
query51	7078	6960	7012	6960
query52	99	88	88	88
query53	250	173	177	173
query54	567	443	453	443
query55	75	71	75	71
query56	282	244	245	244
query57	1202	1057	1097	1057
query58	221	224	251	224
query59	3137	2862	2706	2706
query60	291	263	272	263
query61	105	132	101	101
query62	764	648	657	648
query63	213	186	178	178
query64	2511	680	664	664
query65	3196	3175	3146	3146
query66	677	328	329	328
query67	15525	15152	15134	15134
query68	3009	574	582	574
query69	407	284	322	284
query70	1194	1152	1107	1107
query71	352	275	276	275
query72	5372	4028	4073	4028
query73	753	330	329	329
query74	9162	8869	8889	8869
query75	3340	2642	2706	2642
query76	1411	1047	986	986
query77	571	316	323	316
query78	9715	9135	8971	8971
query79	1027	538	536	536
query80	727	505	509	505
query81	530	237	236	236
query82	250	147	139	139
query83	171	148	148	148
query84	262	78	78	78
query85	685	292	283	283
query86	304	307	300	300
query87	4319	4284	4208	4208
query88	3141	2330	2326	2326
query89	373	288	284	284
query90	1988	189	184	184
query91	136	112	114	112
query92	61	53	54	53
query93	1054	534	534	534
query94	733	292	370	292
query95	327	250	253	250
query96	598	272	268	268
query97	3182	3087	3077	3077
query98	216	200	206	200
query99	1529	1320	1241	1241
Total cold run time: 303172 ms
Total hot run time: 192550 ms

doris-robot · 2024-09-04T04:53:59Z

ClickBench: Total hot run time: 32.1 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7b1cae23c02feafeae3acc0f8c781522235618c0, data reload: false
query1	0.05	0.05	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.67	0.08	0.08
query5	0.51	0.48	0.49
query6	1.13	0.73	0.72
query7	0.02	0.01	0.02
query8	0.06	0.05	0.04
query9	0.55	0.49	0.49
query10	0.55	0.55	0.55
query11	0.15	0.11	0.12
query12	0.14	0.12	0.13
query13	0.62	0.59	0.58
query14	2.06	2.15	2.14
query15	0.84	0.82	0.82
query16	0.37	0.37	0.37
query17	1.02	0.97	0.98
query18	0.22	0.20	0.20
query19	1.82	1.77	1.72
query20	0.01	0.01	0.01
query21	15.40	0.69	0.69
query22	3.97	7.78	1.98
query23	18.30	1.36	1.22
query24	2.20	0.22	0.21
query25	0.16	0.08	0.08
query26	0.26	0.18	0.17
query27	0.08	0.07	0.08
query28	13.22	1.00	0.99
query29	12.66	3.35	3.33
query30	0.24	0.05	0.04
query31	2.88	0.42	0.42
query32	3.23	0.49	0.47
query33	2.96	3.01	3.01
query34	17.14	4.34	4.42
query35	4.52	4.49	4.47
query36	0.67	0.48	0.47
query37	0.18	0.16	0.16
query38	0.16	0.16	0.16
query39	0.05	0.04	0.03
query40	0.16	0.12	0.13
query41	0.10	0.05	0.05
query42	0.06	0.04	0.06
query43	0.04	0.05	0.04
Total cold run time: 110.73 s
Total hot run time: 32.1 s

kaijchen · 2024-09-04T09:00:15Z

run buildall

github-actions · 2024-09-04T09:06:47Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-09-04T10:03:00Z

TeamCity be ut coverage result:
Function Coverage: 36.84% (9394/25497)
Line Coverage: 28.27% (77447/273955)
Region Coverage: 27.67% (39970/144442)
Branch Coverage: 24.32% (20339/83646)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a73d9718c83a99de8fc7f308ac3ae702955bb557_a73d9718c83a99de8fc7f308ac3ae702955bb557/report/index.html

doris-robot · 2024-09-04T10:11:19Z

TPC-H: Total hot run time: 38797 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a73d9718c83a99de8fc7f308ac3ae702955bb557, data reload: false
------ Round 1 ----------------------------------
q1	18226	4869	4421	4421
q2	2657	189	172	172
q3	11100	1138	1272	1138
q4	10581	742	723	723
q5	8325	2981	2903	2903
q6	231	140	150	140
q7	1017	633	630	630
q8	9658	2100	2109	2100
q9	7052	6590	6578	6578
q10	7022	2255	2245	2245
q11	456	245	249	245
q12	401	227	224	224
q13	17770	3072	3069	3069
q14	288	234	244	234
q15	542	520	505	505
q16	576	493	496	493
q17	985	674	685	674
q18	7479	7002	6920	6920
q19	1394	1096	1149	1096
q20	708	330	325	325
q21	4156	3111	2945	2945
q22	1119	1028	1017	1017
Total cold run time: 111743 ms
Total hot run time: 38797 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4357	4316	4349	4316
q2	386	268	284	268
q3	2872	2689	2645	2645
q4	1909	1712	1694	1694
q5	5443	5422	5483	5422
q6	218	133	133	133
q7	2063	1722	1751	1722
q8	3254	3376	3383	3376
q9	8509	8555	8478	8478
q10	3463	3206	3194	3194
q11	651	495	503	495
q12	794	611	586	586
q13	9514	3115	3113	3113
q14	303	280	288	280
q15	529	477	476	476
q16	618	552	560	552
q17	1797	1507	1519	1507
q18	7782	7414	7604	7414
q19	1662	1606	1582	1582
q20	2095	1821	1834	1821
q21	5356	5198	5258	5198
q22	1123	1058	1046	1046
Total cold run time: 64698 ms
Total hot run time: 55318 ms

doris-robot · 2024-09-04T10:23:10Z

TPC-DS: Total hot run time: 188284 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a73d9718c83a99de8fc7f308ac3ae702955bb557, data reload: false
query1	924	376	371	371
query2	6477	1935	1901	1901
query3	6642	208	218	208
query4	34312	23001	23172	23001
query5	4205	502	475	475
query6	256	166	174	166
query7	4582	292	297	292
query8	261	228	212	212
query9	8670	2502	2489	2489
query10	438	273	259	259
query11	15630	15105	15173	15105
query12	146	98	99	98
query13	1640	373	368	368
query14	9681	7184	7147	7147
query15	289	169	178	169
query16	8128	488	480	480
query17	1618	577	567	567
query18	2137	293	294	293
query19	339	151	155	151
query20	118	113	110	110
query21	214	105	106	105
query22	4526	4067	4189	4067
query23	33835	33669	34931	33669
query24	12370	2883	2805	2805
query25	550	404	408	404
query26	1169	162	158	158
query27	2815	285	283	283
query28	7330	2092	2071	2071
query29	678	433	441	433
query30	316	169	154	154
query31	989	790	793	790
query32	102	53	64	53
query33	796	300	291	291
query34	972	479	485	479
query35	874	748	720	720
query36	1095	913	921	913
query37	150	95	90	90
query38	3964	3897	4036	3897
query39	1454	1380	1377	1377
query40	273	120	119	119
query41	51	48	49	48
query42	117	101	101	101
query43	504	479	453	453
query44	1216	759	747	747
query45	202	167	170	167
query46	1096	745	754	745
query47	1906	1789	1845	1789
query48	380	299	304	299
query49	1161	435	445	435
query50	811	426	416	416
query51	7102	6884	6792	6792
query52	98	86	91	86
query53	263	190	185	185
query54	921	465	462	462
query55	82	78	76	76
query56	279	259	261	259
query57	1203	1117	1085	1085
query58	258	233	247	233
query59	3007	2660	2768	2660
query60	302	281	275	275
query61	122	118	117	117
query62	862	617	650	617
query63	223	195	190	190
query64	5133	799	671	671
query65	3252	3114	3195	3114
query66	1277	354	331	331
query67	15675	15410	15575	15410
query68	4806	576	566	566
query69	418	310	284	284
query70	1132	1140	1096	1096
query71	452	280	283	280
query72	6305	3926	4011	3926
query73	752	331	330	330
query74	9329	8886	8892	8886
query75	3402	2651	2706	2651
query76	2825	1074	1001	1001
query77	480	326	324	324
query78	11145	9526	9026	9026
query79	2130	528	543	528
query80	813	521	513	513
query81	592	236	230	230
query82	557	140	149	140
query83	250	148	154	148
query84	229	77	77	77
query85	713	323	274	274
query86	468	303	265	265
query87	4406	4352	4330	4330
query88	3643	2341	2350	2341
query89	391	287	282	282
query90	1898	190	184	184
query91	123	102	100	100
query92	67	52	50	50
query93	1731	558	552	552
query94	826	290	281	281
query95	347	255	256	255
query96	587	276	263	263
query97	3200	3126	3129	3126
query98	219	212	200	200
query99	1601	1348	1272	1272
Total cold run time: 294305 ms
Total hot run time: 188284 ms

kaijchen · 2024-09-05T02:03:53Z

run buildall

doris-robot · 2024-09-05T03:51:36Z

TPC-H: Total hot run time: 38143 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6b36583a86c8d7b209657d31d48e53766328b671, data reload: false
------ Round 1 ----------------------------------
q1	18060	4545	4348	4348
q2	2037	190	170	170
q3	11238	1029	1001	1001
q4	10141	770	752	752
q5	7759	2894	2815	2815
q6	233	141	139	139
q7	975	623	606	606
q8	9339	2075	2050	2050
q9	7287	6568	6574	6568
q10	7024	2186	2211	2186
q11	477	243	237	237
q12	392	228	231	228
q13	17758	3110	3142	3110
q14	270	242	235	235
q15	529	495	496	495
q16	611	527	516	516
q17	999	663	723	663
q18	7328	6827	6915	6827
q19	1472	990	1033	990
q20	708	326	331	326
q21	3983	2949	2872	2872
q22	1086	1034	1009	1009
Total cold run time: 109706 ms
Total hot run time: 38143 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4432	4376	4328	4328
q2	384	275	279	275
q3	2885	2685	2669	2669
q4	1980	1660	1666	1660
q5	5626	5718	5691	5691
q6	236	146	142	142
q7	2236	1782	1810	1782
q8	3312	3421	3445	3421
q9	8883	8913	8821	8821
q10	3540	3360	3367	3360
q11	617	509	528	509
q12	827	660	648	648
q13	14582	3258	3275	3258
q14	332	302	311	302
q15	534	476	495	476
q16	634	611	580	580
q17	1847	1564	1547	1547
q18	8116	7971	7790	7790
q19	1740	1654	1707	1654
q20	2234	1951	1878	1878
q21	5843	5510	5515	5510
q22	1116	1059	1001	1001
Total cold run time: 71936 ms
Total hot run time: 57302 ms

doris-robot · 2024-09-05T04:00:10Z

TeamCity be ut coverage result:
Function Coverage: 36.84% (9392/25497)
Line Coverage: 28.26% (77421/273923)
Region Coverage: 27.66% (39952/144439)
Branch Coverage: 24.30% (20327/83646)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6b36583a86c8d7b209657d31d48e53766328b671_6b36583a86c8d7b209657d31d48e53766328b671/report/index.html

doris-robot · 2024-09-05T04:03:53Z

TPC-DS: Total hot run time: 194854 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6b36583a86c8d7b209657d31d48e53766328b671, data reload: false
query1	1272	908	871	871
query2	6314	1863	1947	1863
query3	10603	4001	4042	4001
query4	59666	23991	23517	23517
query5	5359	502	506	502
query6	405	177	164	164
query7	5798	305	293	293
query8	292	206	204	204
query9	8877	2530	2519	2519
query10	498	281	284	281
query11	17763	15236	15647	15236
query12	154	106	103	103
query13	1589	397	388	388
query14	11256	7858	7799	7799
query15	251	171	185	171
query16	7594	450	457	450
query17	1125	579	573	573
query18	1595	303	286	286
query19	315	147	148	147
query20	120	112	111	111
query21	211	113	115	113
query22	4709	4363	4650	4363
query23	34449	33615	33436	33436
query24	5988	2942	2898	2898
query25	527	381	379	379
query26	691	158	156	156
query27	1796	283	279	279
query28	3865	2073	2057	2057
query29	664	415	415	415
query30	239	154	160	154
query31	926	749	765	749
query32	81	50	52	50
query33	495	280	293	280
query34	907	488	495	488
query35	856	732	702	702
query36	1064	914	946	914
query37	150	89	96	89
query38	4030	3889	3947	3889
query39	1460	1398	1395	1395
query40	200	115	115	115
query41	47	46	44	44
query42	117	98	100	98
query43	542	480	491	480
query44	1137	738	751	738
query45	195	163	165	163
query46	1113	753	750	750
query47	1874	1826	1830	1826
query48	393	295	300	295
query49	782	434	439	434
query50	845	444	422	422
query51	6925	6890	6820	6820
query52	105	87	94	87
query53	261	181	182	181
query54	580	457	465	457
query55	84	77	78	77
query56	323	260	260	260
query57	1209	1084	1060	1060
query58	222	226	226	226
query59	3136	2810	2698	2698
query60	308	272	284	272
query61	103	96	93	93
query62	776	670	670	670
query63	214	182	189	182
query64	2746	691	770	691
query65	3293	3195	3217	3195
query66	691	361	357	357
query67	15315	15241	15252	15241
query68	2988	591	614	591
query69	415	305	301	301
query70	1173	1114	1118	1114
query71	361	294	286	286
query72	5453	4216	4119	4119
query73	783	336	333	333
query74	9426	8927	8830	8830
query75	3450	2713	2757	2713
query76	1452	1004	1058	1004
query77	563	341	342	341
query78	11386	9489	9383	9383
query79	1124	573	542	542
query80	768	557	557	557
query81	552	245	243	243
query82	338	153	156	153
query83	179	158	157	157
query84	270	86	85	85
query85	762	355	336	336
query86	321	297	270	270
query87	4414	4400	4338	4338
query88	3452	2584	2415	2415
query89	400	290	287	287
query90	2095	193	192	192
query91	149	102	100	100
query92	63	49	54	49
query93	1095	548	549	548
query94	772	277	297	277
query95	353	254	250	250
query96	603	271	274	271
query97	3182	3078	3086	3078
query98	214	212	205	205
query99	1692	1269	1269	1269
Total cold run time: 307738 ms
Total hot run time: 194854 ms

doris-robot · 2024-09-05T04:09:31Z

ClickBench: Total hot run time: 31.89 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6b36583a86c8d7b209657d31d48e53766328b671, data reload: false
query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.08	0.09
query5	0.51	0.51	0.49
query6	1.13	0.72	0.73
query7	0.02	0.02	0.01
query8	0.05	0.05	0.05
query9	0.56	0.49	0.48
query10	0.52	0.54	0.54
query11	0.15	0.12	0.11
query12	0.15	0.13	0.12
query13	0.62	0.59	0.58
query14	2.05	2.02	2.07
query15	0.84	0.82	0.81
query16	0.35	0.37	0.39
query17	1.01	0.98	1.00
query18	0.22	0.21	0.21
query19	1.78	1.74	1.77
query20	0.02	0.01	0.02
query21	15.40	0.68	0.66
query22	4.06	7.44	1.93
query23	18.26	1.46	1.25
query24	2.10	0.22	0.22
query25	0.15	0.08	0.08
query26	0.26	0.18	0.18
query27	0.08	0.07	0.07
query28	13.22	1.02	1.00
query29	12.62	3.32	3.33
query30	0.24	0.06	0.05
query31	2.91	0.40	0.40
query32	3.22	0.47	0.48
query33	2.94	2.99	2.99
query34	16.99	4.45	4.36
query35	4.43	4.41	4.51
query36	0.67	0.46	0.47
query37	0.19	0.16	0.16
query38	0.15	0.16	0.14
query39	0.04	0.04	0.04
query40	0.15	0.12	0.13
query41	0.10	0.04	0.04
query42	0.06	0.06	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.29 s
Total hot run time: 31.89 s

liaoxin01

LGTM

github-actions · 2024-09-05T13:09:40Z

PR approved by at least one committer and no changes requested.

dataroaring

LGTM

dataroaring

LGTM

) backport #40325

) ## Proposed changes This PR is a followup of #40325. Because PipelineX has deprecated the old report. This PR fixed the `ScannedRows` and `LoadBytes` in the progress of the `SHOW LOAD` result. Currently the progress will only be updated when a fragment instance is done. Timely progress updates will be supported in a later PR.

…che#40694) This PR is a followup of apache#40325. Because PipelineX has deprecated the old report. This PR fixed the `ScannedRows` and `LoadBytes` in the progress of the `SHOW LOAD` result. Currently the progress will only be updated when a fragment instance is done. Timely progress updates will be supported in a later PR.

…che#40694) ## Proposed changes This PR is a followup of apache#40325. Because PipelineX has deprecated the old report. This PR fixed the `ScannedRows` and `LoadBytes` in the progress of the `SHOW LOAD` result. Currently the progress will only be updated when a fragment instance is done. Timely progress updates will be supported in a later PR.

dataroaring previously approved these changes Sep 3, 2024
View reviewed changes

dataroaring added the dev/2.1.x label Sep 3, 2024

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 3, 2024

github-actionsBot added the reviewed label Sep 3, 2024

kaijchen added a commit to kaijchen/doris that referenced this pull request Sep 3, 2024
[fix](load) fix broker load progress (apache#40325)
d508972

dataroaring added the dev/2.0.x label Sep 3, 2024

kaijchen added a commit to kaijchen/doris that referenced this pull request Sep 3, 2024
[fix](load) fix broker load progress (apache#40325)
8aa87bb

[fix](pipelinex) fix fragment instance progress reports
7b1cae2

kaijchen dismissed dataroaring’s stale review via 7b1cae2September 4, 2024 01:58

kaijchen force-pushed the DORIS-12389 branch from 745d41d to 7b1cae2Compare September 4, 2024 01:58

kaijchen changed the title ~~[fix](load) fix broker load progress~~[fix](pipelinex) fix fragment instance progress reportsSep 4, 2024

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Sep 4, 2024

Merge branch 'master' into DORIS-12389
a73d971

Merge branch 'master' into DORIS-12389
6b36583

liaoxin01 approved these changes Sep 5, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 5, 2024

dataroaring approved these changes Sep 5, 2024
View reviewed changes

dataroaring added the dev/3.0.x label Sep 5, 2024

dataroaring approved these changes Sep 5, 2024
View reviewed changes

dataroaring merged commit 34f04ee into apache:masterSep 5, 2024

dataroaring added dev/3.0.2-merged and removed dev/3.0.x labels Sep 5, 2024

kaijchen mentioned this pull request Sep 11, 2024
[fix](pipelinex) fix fragment instance progress reports (part 2) #40694
Merged

kaijchen mentioned this pull request Sep 19, 2024
[fix](pipelinex) fix fragment instance progress reports (#40325) #40987
Merged

yiguolei pushed a commit that referenced this pull request Sep 19, 2024
[fix](pipelinex) fix fragment instance progress reports (#40325) (#40987
549bc3e
) backport #40325

yiguolei added dev/2.1.7-merged and removed dev/2.1.x labels Sep 19, 2024

gavinchou mentioned this pull request Oct 13, 2024
Release Note 3.0.2 #41558
Closed

liaoxin01 removed the dev/2.0.x label Oct 22, 2024

Uh oh!

Conversation

kaijchen commented Sep 3, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Uh oh!

doris-robot commented Sep 3, 2024

Uh oh!

kaijchen commented Sep 3, 2024

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Sep 3, 2024

Uh oh!

github-actionsBot commented Sep 3, 2024

Uh oh!

doris-robot commented Sep 3, 2024

Uh oh!

doris-robot commented Sep 3, 2024

Uh oh!

doris-robot commented Sep 3, 2024

Uh oh!

github-actionsBot commented Sep 4, 2024

Uh oh!

kaijchen commented Sep 4, 2024

Uh oh!

doris-robot commented Sep 4, 2024

Uh oh!

doris-robot commented Sep 4, 2024

Uh oh!

doris-robot commented Sep 4, 2024

Uh oh!

kaijchen commented Sep 4, 2024

Uh oh!

github-actionsBot commented Sep 4, 2024

Uh oh!

doris-robot commented Sep 4, 2024

Uh oh!

doris-robot commented Sep 4, 2024

Uh oh!

doris-robot commented Sep 4, 2024

Uh oh!

kaijchen commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

liaoxin01 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Sep 5, 2024

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kaijchen commented Sep 3, 2024•
edited
Loading