[improvement](cloud) manage node via sql like non cloud mode by dataroaring · Pull Request #40264 · apache/doris

dataroaring · 2024-09-02T09:35:11Z

Proposed changes

add a config deploy_mode to enable disaggregated mode. In this mode users have to config cloud_instance_id and meta_service_endpoint.
When a fe starts from empty, it would try to create instance.
If the instance does not exists, it creates and starts from master role. If the instance exists, then it stats with its role got from ms.
Frontends are added via sql alter system add frontend. Backends are added via sql alter system add backend.
Users do not need config cloud_instance_id and meta_service_endpoint in be.conf, because fe sends them to be via heartbeat.
Builtin vault is not needed any more, internal tabels are stored in default vault.

TODO:

decomission fe and be via sql
change cloud_instance_id to cluster id.

doc pr apache/doris-website#1072

doris-robot · 2024-09-02T09:35:16Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

github-actions

clang-tidy made some suggestions

github-actions

clang-tidy made some suggestions

dataroaring · 2024-09-02T15:21:46Z

run buildall

dataroaring · 2024-09-09T08:05:36Z

run buildall

dataroaring · 2024-09-09T08:27:40Z

run buildall

dataroaring · 2024-09-09T09:21:32Z

run buildall

doris-robot · 2024-09-09T10:59:53Z

TeamCity be ut coverage result:
Function Coverage: 36.84% (9386/25479)
Line Coverage: 28.24% (77421/274173)
Region Coverage: 27.64% (39976/144654)
Branch Coverage: 24.27% (20340/83818)
Coverage Report: http://coverage.selectdb-in.cc/coverage/64d54ff8a19491d33ee4fb2708441b3eb54b1922_64d54ff8a19491d33ee4fb2708441b3eb54b1922/report/index.html

dataroaring · 2024-09-09T13:12:34Z

run buildall

doris-robot · 2024-09-09T13:47:13Z

TeamCity be ut coverage result:
Function Coverage: 36.84% (9386/25479)
Line Coverage: 28.24% (77415/274173)
Region Coverage: 27.64% (39983/144654)
Branch Coverage: 24.27% (20343/83818)
Coverage Report: http://coverage.selectdb-in.cc/coverage/5767a7d0811e6b38e7c27d9701d4c8a13942be65_5767a7d0811e6b38e7c27d9701d4c8a13942be65/report/index.html

doris-robot · 2024-09-09T14:18:10Z

TPC-H: Total hot run time: 38370 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5767a7d0811e6b38e7c27d9701d4c8a13942be65, data reload: false
------ Round 1 ----------------------------------
q1	18203	4634	4436	4436
q2	2020	192	182	182
q3	10486	1137	1064	1064
q4	10144	746	692	692
q5	7757	2883	2843	2843
q6	224	141	143	141
q7	971	618	615	615
q8	9320	2128	2062	2062
q9	7068	6592	6577	6577
q10	7011	2175	2270	2175
q11	439	252	256	252
q12	416	239	235	235
q13	18062	3163	3140	3140
q14	294	239	243	239
q15	549	493	496	493
q16	536	456	444	444
q17	989	695	670	670
q18	7563	6948	6814	6814
q19	1389	1030	989	989
q20	669	350	330	330
q21	4140	3026	2995	2995
q22	1136	982	985	982
Total cold run time: 109386 ms
Total hot run time: 38370 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4378	4320	4299	4299
q2	387	274	264	264
q3	2926	2727	2718	2718
q4	2038	1672	1597	1597
q5	5404	5407	5470	5407
q6	222	134	138	134
q7	2125	1782	1749	1749
q8	3250	3370	3356	3356
q9	8463	8517	8484	8484
q10	3457	3199	3234	3199
q11	617	499	509	499
q12	823	597	611	597
q13	13914	3131	3172	3131
q14	312	274	277	274
q15	526	494	480	480
q16	533	478	481	478
q17	1796	1508	1509	1508
q18	7743	7597	7443	7443
q19	1690	1551	1592	1551
q20	2064	1876	1809	1809
q21	5521	5307	5176	5176
q22	1121	1038	1032	1032
Total cold run time: 69310 ms
Total hot run time: 55185 ms

doris-robot · 2024-09-09T14:29:52Z

TPC-DS: Total hot run time: 187461 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5767a7d0811e6b38e7c27d9701d4c8a13942be65, data reload: false
query1	920	382	398	382
query2	6495	1905	1915	1905
query3	6651	209	215	209
query4	33706	23133	23378	23133
query5	4171	504	495	495
query6	274	169	167	167
query7	4585	299	293	293
query8	284	232	224	224
query9	8496	2486	2481	2481
query10	450	277	285	277
query11	16892	15184	15032	15032
query12	162	105	100	100
query13	1641	402	382	382
query14	9994	7270	7204	7204
query15	261	163	181	163
query16	8077	457	484	457
query17	1643	562	550	550
query18	2064	290	283	283
query19	335	143	140	140
query20	115	109	108	108
query21	207	103	102	102
query22	4455	4013	4120	4013
query23	33846	33218	33138	33138
query24	11162	2932	2844	2844
query25	614	369	384	369
query26	1316	154	158	154
query27	2807	279	278	278
query28	7656	2011	2015	2011
query29	877	418	407	407
query30	308	159	151	151
query31	956	746	810	746
query32	100	57	61	57
query33	754	293	289	289
query34	968	484	497	484
query35	866	728	718	718
query36	1068	929	903	903
query37	150	82	86	82
query38	4011	3849	3976	3849
query39	1475	1400	1402	1400
query40	256	113	113	113
query41	51	45	44	44
query42	117	92	95	92
query43	502	475	484	475
query44	1242	778	757	757
query45	196	164	172	164
query46	1109	765	773	765
query47	1895	1757	1786	1757
query48	403	308	302	302
query49	1047	433	429	429
query50	832	417	427	417
query51	7004	7011	6844	6844
query52	97	86	90	86
query53	257	193	186	186
query54	918	472	462	462
query55	74	76	76	76
query56	276	257	259	257
query57	1213	1073	1086	1073
query58	247	237	248	237
query59	2985	2846	2692	2692
query60	290	267	264	264
query61	134	100	102	100
query62	814	659	642	642
query63	220	186	186	186
query64	4618	673	631	631
query65	3216	3156	3157	3156
query66	1418	333	339	333
query67	15835	15330	15139	15139
query68	3551	595	578	578
query69	404	282	295	282
query70	1188	1045	1092	1045
query71	346	328	262	262
query72	6344	4101	4063	4063
query73	737	329	335	329
query74	9146	8908	8776	8776
query75	3350	2735	2719	2719
query76	1901	975	986	975
query77	458	314	328	314
query78	9781	9037	9052	9037
query79	1025	543	530	530
query80	699	532	507	507
query81	451	229	232	229
query82	233	139	138	138
query83	171	154	161	154
query84	222	85	77	77
query85	680	290	277	277
query86	327	305	301	301
query87	4360	4358	4282	4282
query88	3362	2456	2389	2389
query89	370	287	294	287
query90	1872	199	199	199
query91	128	100	104	100
query92	62	51	49	49
query93	1034	550	549	549
query94	714	295	289	289
query95	368	262	257	257
query96	586	274	269	269
query97	3215	3111	3049	3049
query98	217	207	203	203
query99	1497	1287	1296	1287
Total cold run time: 286877 ms
Total hot run time: 187461 ms

doris-robot · 2024-09-09T14:35:21Z

ClickBench: Total hot run time: 32.71 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5767a7d0811e6b38e7c27d9701d4c8a13942be65, data reload: false
query1	0.05	0.05	0.04
query2	0.08	0.03	0.04
query3	0.22	0.05	0.05
query4	1.67	0.10	0.10
query5	0.52	0.51	0.51
query6	1.12	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.54	0.49	0.47
query10	0.54	0.56	0.55
query11	0.16	0.12	0.12
query12	0.15	0.13	0.12
query13	0.60	0.58	0.59
query14	1.38	1.44	1.44
query15	0.85	0.84	0.83
query16	0.37	0.37	0.38
query17	1.03	1.03	1.01
query18	0.22	0.20	0.21
query19	1.94	1.86	1.87
query20	0.02	0.01	0.02
query21	15.40	0.66	0.67
query22	3.74	6.89	2.96
query23	18.27	1.40	1.35
query24	2.14	0.23	0.21
query25	0.16	0.08	0.08
query26	0.27	0.18	0.18
query27	0.07	0.07	0.07
query28	13.20	1.02	0.99
query29	12.61	3.35	3.42
query30	0.24	0.05	0.06
query31	2.87	0.41	0.42
query32	3.24	0.48	0.47
query33	2.97	3.03	3.03
query34	17.08	4.36	4.39
query35	4.42	4.41	4.42
query36	0.65	0.50	0.49
query37	0.18	0.15	0.15
query38	0.16	0.15	0.15
query39	0.04	0.03	0.04
query40	0.17	0.12	0.14
query41	0.10	0.04	0.05
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.61 s
Total hot run time: 32.71 s

dataroaring · 2024-09-09T15:49:26Z

run buildall

doris-robot · 2024-09-09T16:13:01Z

TeamCity be ut coverage result:
Function Coverage: 36.83% (9385/25479)
Line Coverage: 28.22% (77375/274173)
Region Coverage: 27.63% (39966/144654)
Branch Coverage: 24.26% (20334/83818)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3a1bfb28dc6d59c11c7ca5adb365ed395e5ab306_3a1bfb28dc6d59c11c7ca5adb365ed395e5ab306/report/index.html

doris-robot · 2024-09-09T16:42:20Z

TPC-H: Total hot run time: 38003 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3a1bfb28dc6d59c11c7ca5adb365ed395e5ab306, data reload: false
------ Round 1 ----------------------------------
q1	18892	7206	4349	4349
q2	2031	192	184	184
q3	11579	972	1040	972
q4	10486	863	738	738
q5	7754	2851	2872	2851
q6	231	139	141	139
q7	971	618	614	614
q8	9331	2085	2082	2082
q9	7131	6550	6635	6550
q10	7005	2231	2288	2231
q11	522	248	240	240
q12	401	223	228	223
q13	17759	3042	3097	3042
q14	277	234	234	234
q15	549	482	484	482
q16	519	421	446	421
q17	979	619	638	619
q18	7384	6762	6831	6762
q19	1391	956	981	956
q20	693	329	336	329
q21	3892	3031	2991	2991
q22	1085	997	994	994
Total cold run time: 110862 ms
Total hot run time: 38003 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4397	4310	4290	4290
q2	375	263	283	263
q3	2910	2686	2690	2686
q4	1892	1673	1723	1673
q5	5663	5689	5801	5689
q6	228	132	136	132
q7	2239	1822	1798	1798
q8	3304	3500	3486	3486
q9	8829	8797	8737	8737
q10	3641	3415	3321	3321
q11	614	500	520	500
q12	817	647	655	647
q13	14640	3179	3275	3179
q14	321	293	283	283
q15	535	505	490	490
q16	551	487	513	487
q17	1841	1572	1525	1525
q18	8229	7989	7832	7832
q19	1768	1457	1677	1457
q20	2159	1905	1894	1894
q21	5771	5548	5386	5386
q22	1108	1044	1028	1028
Total cold run time: 71832 ms
Total hot run time: 56783 ms

doris-robot · 2024-09-09T16:54:35Z

TPC-DS: Total hot run time: 192166 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3a1bfb28dc6d59c11c7ca5adb365ed395e5ab306, data reload: false
query1	1252	884	835	835
query2	6373	1889	1899	1889
query3	10606	4125	3886	3886
query4	59453	25570	23032	23032
query5	5401	509	507	507
query6	401	180	163	163
query7	5780	309	299	299
query8	306	217	216	216
query9	9103	2485	2474	2474
query10	484	289	261	261
query11	17953	14997	15253	14997
query12	160	109	106	106
query13	1570	409	414	409
query14	11088	7150	7127	7127
query15	265	172	171	171
query16	7589	480	475	475
query17	1189	569	557	557
query18	2026	291	307	291
query19	299	160	168	160
query20	135	110	109	109
query21	217	109	107	107
query22	4539	4504	4571	4504
query23	34169	33717	34566	33717
query24	5951	2826	2849	2826
query25	511	411	407	407
query26	682	167	156	156
query27	1750	281	287	281
query28	3845	2016	2022	2016
query29	656	431	428	428
query30	244	152	152	152
query31	948	757	802	757
query32	86	54	59	54
query33	440	296	289	289
query34	897	490	474	474
query35	847	726	700	700
query36	1061	952	944	944
query37	147	92	89	89
query38	3994	3996	3840	3840
query39	1437	1391	1368	1368
query40	201	115	115	115
query41	49	51	46	46
query42	116	98	93	93
query43	512	486	484	484
query44	1110	760	764	760
query45	197	164	164	164
query46	1091	768	755	755
query47	1904	1800	1831	1800
query48	377	308	297	297
query49	790	451	462	451
query50	831	409	404	404
query51	6941	6835	6858	6835
query52	103	86	86	86
query53	249	178	182	178
query54	595	463	470	463
query55	73	75	74	74
query56	291	288	274	274
query57	1221	1067	1052	1052
query58	238	250	244	244
query59	2966	2780	2696	2696
query60	309	276	359	276
query61	104	97	100	97
query62	740	660	660	660
query63	212	182	183	182
query64	2788	694	663	663
query65	3223	3125	3152	3125
query66	616	333	340	333
query67	15374	15278	15154	15154
query68	4023	584	573	573
query69	402	275	282	275
query70	1086	1111	1091	1091
query71	340	275	278	275
query72	6262	4035	3916	3916
query73	755	323	328	323
query74	9176	8782	8698	8698
query75	3351	2691	2739	2691
query76	1706	1013	963	963
query77	591	315	314	314
query78	9874	9571	9047	9047
query79	1346	533	526	526
query80	873	527	496	496
query81	522	226	226	226
query82	753	135	147	135
query83	167	146	142	142
query84	261	73	75	73
query85	794	278	273	273
query86	321	291	293	291
query87	4349	4385	4205	4205
query88	2954	2427	2379	2379
query89	388	285	282	282
query90	1787	183	185	183
query91	123	94	96	94
query92	63	49	51	49
query93	1488	535	535	535
query94	708	299	281	281
query95	343	250	257	250
query96	603	263	265	263
query97	3196	3064	3047	3047
query98	221	204	201	201
query99	1791	1235	1248	1235
Total cold run time: 307692 ms
Total hot run time: 192166 ms

doris-robot · 2024-09-09T17:00:10Z

ClickBench: Total hot run time: 31.89 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3a1bfb28dc6d59c11c7ca5adb365ed395e5ab306, data reload: false
query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.22	0.04	0.05
query4	1.67	0.08	0.09
query5	0.51	0.52	0.49
query6	1.13	0.74	0.73
query7	0.01	0.01	0.01
query8	0.06	0.05	0.05
query9	0.55	0.50	0.49
query10	0.54	0.56	0.54
query11	0.16	0.12	0.12
query12	0.14	0.12	0.11
query13	0.60	0.59	0.57
query14	1.39	1.43	1.44
query15	0.83	0.84	0.82
query16	0.38	0.37	0.38
query17	1.08	1.00	1.04
query18	0.21	0.20	0.20
query19	1.94	1.85	1.77
query20	0.01	0.01	0.02
query21	15.40	0.66	0.65
query22	4.01	7.15	2.46
query23	18.32	1.31	1.23
query24	2.10	0.24	0.21
query25	0.15	0.08	0.07
query26	0.26	0.18	0.17
query27	0.08	0.07	0.08
query28	13.25	1.00	0.99
query29	12.62	3.41	3.41
query30	0.24	0.05	0.07
query31	2.88	0.40	0.39
query32	3.25	0.48	0.47
query33	2.93	2.96	2.99
query34	17.01	4.36	4.36
query35	4.43	4.45	4.41
query36	0.65	0.47	0.49
query37	0.19	0.16	0.16
query38	0.16	0.15	0.14
query39	0.04	0.04	0.04
query40	0.16	0.13	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.88 s
Total hot run time: 31.89 s

doris-robot · 2024-09-11T18:16:15Z

TPC-H: Total hot run time: 38747 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 64dd69b66d9c2bd2531b32c29f88a482ea7723d0, data reload: false
------ Round 1 ----------------------------------
q1	17993	4691	4455	4455
q2	2050	189	187	187
q3	11038	989	1106	989
q4	10516	680	755	680
q5	7773	2947	2883	2883
q6	231	142	138	138
q7	972	629	613	613
q8	9340	2088	2121	2088
q9	7270	6632	6633	6632
q10	6994	2256	2229	2229
q11	445	240	249	240
q12	402	226	221	221
q13	17759	3097	3124	3097
q14	287	239	259	239
q15	536	514	491	491
q16	546	439	435	435
q17	1000	746	689	689
q18	7625	6987	7122	6987
q19	1436	1057	1066	1057
q20	698	334	333	333
q21	4074	3200	3031	3031
q22	1130	1040	1033	1033
Total cold run time: 110115 ms
Total hot run time: 38747 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4457	4381	4310	4310
q2	405	270	278	270
q3	2927	2725	2662	2662
q4	1989	1724	1659	1659
q5	5745	5800	5837	5800
q6	235	140	144	140
q7	2241	1816	1793	1793
q8	3318	3443	3468	3443
q9	8964	8917	8923	8917
q10	3662	3475	3372	3372
q11	614	516	520	516
q12	826	673	656	656
q13	13363	3328	3256	3256
q14	305	296	304	296
q15	540	503	495	495
q16	567	491	498	491
q17	1851	1556	1562	1556
q18	8350	8059	7992	7992
q19	1776	1653	1537	1537
q20	2172	1925	1893	1893
q21	5811	5603	5583	5583
q22	1149	1043	1024	1024
Total cold run time: 71267 ms
Total hot run time: 57661 ms

doris-robot · 2024-09-11T18:28:38Z

TPC-DS: Total hot run time: 199431 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 64dd69b66d9c2bd2531b32c29f88a482ea7723d0, data reload: false
query1	1274	893	864	864
query2	6290	1941	1886	1886
query3	10666	4023	4080	4023
query4	59924	24561	23346	23346
query5	5010	501	496	496
query6	404	189	174	174
query7	5621	292	290	290
query8	327	233	218	218
query9	7804	2516	2494	2494
query10	425	284	277	277
query11	17368	15277	15326	15277
query12	158	104	105	104
query13	1459	392	405	392
query14	9675	7667	7146	7146
query15	204	177	169	169
query16	6774	479	496	479
query17	1103	593	625	593
query18	1474	300	295	295
query19	195	149	158	149
query20	127	107	109	107
query21	220	103	106	103
query22	4908	4519	4703	4519
query23	34592	33988	33892	33892
query24	6039	2898	2942	2898
query25	493	385	384	384
query26	598	148	154	148
query27	1586	278	280	278
query28	3800	2046	2025	2025
query29	646	401	401	401
query30	233	156	167	156
query31	954	757	780	757
query32	78	56	56	56
query33	446	306	299	299
query34	881	490	482	482
query35	879	737	726	726
query36	1072	946	966	946
query37	153	90	89	89
query38	4077	3899	3923	3899
query39	1476	1421	1408	1408
query40	212	116	116	116
query41	49	49	46	46
query42	118	100	96	96
query43	523	483	478	478
query44	1104	752	750	750
query45	202	172	175	172
query46	1098	751	787	751
query47	1987	1847	1865	1847
query48	387	288	296	288
query49	795	488	457	457
query50	845	420	430	420
query51	7050	7020	6986	6986
query52	106	88	89	88
query53	256	187	191	187
query54	572	466	471	466
query55	76	79	79	79
query56	300	271	281	271
query57	1218	1068	1092	1068
query58	250	253	243	243
query59	2945	2832	2911	2832
query60	304	284	291	284
query61	150	124	124	124
query62	766	663	659	659
query63	224	189	191	189
query64	1460	786	759	759
query65	3236	3184	3160	3160
query66	697	345	360	345
query67	16074	15885	15563	15563
query68	1663	552	532	532
query69	421	284	283	283
query70	1214	1136	1140	1136
query71	344	269	271	269
query72	5511	4267	3977	3977
query73	750	320	324	320
query74	9256	8950	9003	8950
query75	3406	2714	2694	2694
query76	1382	955	973	955
query77	514	323	323	323
query78	11372	10261	9697	9697
query79	981	872	872	872
query80	1003	832	828	828
query81	543	263	262	262
query82	790	270	270	270
query83	252	194	183	183
query84	264	105	104	104
query85	730	399	400	399
query86	327	325	335	325
query87	4490	4367	4460	4367
query88	4385	4104	4071	4071
query89	395	365	366	365
query90	1882	315	316	315
query91	125	124	126	124
query92	81	83	79	79
query93	940	906	896	896
query94	810	354	370	354
query95	453	419	428	419
query96	469	473	468	468
query97	3195	3136	3138	3136
query98	236	227	232	227
query99	1507	1298	1262	1262
Total cold run time: 302108 ms
Total hot run time: 199431 ms

doris-robot · 2024-09-11T18:34:13Z

ClickBench: Total hot run time: 32.08 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 64dd69b66d9c2bd2531b32c29f88a482ea7723d0, data reload: false
query1	0.04	0.04	0.04
query2	0.09	0.04	0.04
query3	0.23	0.05	0.05
query4	1.66	0.09	0.10
query5	0.51	0.52	0.51
query6	1.12	0.73	0.74
query7	0.02	0.02	0.01
query8	0.04	0.06	0.04
query9	0.55	0.51	0.50
query10	0.55	0.54	0.53
query11	0.15	0.12	0.12
query12	0.15	0.12	0.12
query13	0.59	0.59	0.59
query14	1.40	1.41	1.40
query15	0.85	0.84	0.81
query16	0.37	0.38	0.37
query17	0.97	1.06	1.00
query18	0.20	0.18	0.17
query19	1.92	1.83	1.88
query20	0.02	0.01	0.01
query21	15.40	0.68	0.69
query22	4.13	6.86	2.34
query23	18.29	1.34	1.27
query24	2.11	0.23	0.21
query25	0.14	0.08	0.07
query26	0.27	0.18	0.18
query27	0.08	0.08	0.08
query28	13.25	1.04	1.01
query29	12.65	3.42	3.39
query30	0.24	0.06	0.05
query31	2.87	0.40	0.39
query32	3.26	0.48	0.48
query33	3.00	3.03	3.04
query34	17.08	4.39	4.41
query35	4.45	4.47	4.52
query36	0.66	0.51	0.47
query37	0.18	0.17	0.15
query38	0.16	0.16	0.16
query39	0.05	0.04	0.05
query40	0.16	0.14	0.13
query41	0.09	0.04	0.04
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.06 s
Total hot run time: 32.08 s

dataroaring · 2024-09-12T05:44:03Z

run buildall

doris-robot · 2024-09-12T06:08:58Z

TPC-H: Total hot run time: 40423 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4e88974b17a72cd2c3476e25a5d2830cd7c5b50c, data reload: false
------ Round 1 ----------------------------------
q1	17621	4494	4228	4228
q2	2025	219	216	216
q3	10837	1468	1431	1431
q4	11083	1005	1060	1005
q5	8056	3287	3216	3216
q6	235	138	141	138
q7	1070	658	632	632
q8	9529	2062	2075	2062
q9	7008	6424	6428	6424
q10	7392	2535	2579	2535
q11	437	247	255	247
q12	416	233	229	229
q13	17746	3062	3048	3048
q14	283	247	253	247
q15	559	497	495	495
q16	518	435	432	432
q17	984	953	942	942
q18	7465	6856	6802	6802
q19	1398	1235	1252	1235
q20	615	332	343	332
q21	3936	3553	3536	3536
q22	1118	991	1037	991
Total cold run time: 110331 ms
Total hot run time: 40423 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4273	4237	4192	4192
q2	350	241	232	232
q3	2899	2896	2908	2896
q4	1951	2082	2031	2031
q5	5907	5665	5615	5615
q6	228	132	140	132
q7	2176	1836	1793	1793
q8	3366	3488	3433	3433
q9	8901	8831	8841	8831
q10	3503	3542	3548	3542
q11	590	475	467	467
q12	845	591	664	591
q13	4696	3256	3192	3192
q14	323	279	278	278
q15	556	523	508	508
q16	513	477	469	469
q17	1814	1755	1772	1755
q18	8483	8165	8097	8097
q19	1735	1713	1720	1713
q20	2098	1875	1862	1862
q21	5920	5645	5640	5640
q22	1175	1068	1051	1051
Total cold run time: 62302 ms
Total hot run time: 58320 ms

doris-robot · 2024-09-12T06:20:28Z

TeamCity be ut coverage result:
Function Coverage: 36.89% (9455/25630)
Line Coverage: 28.26% (77788/275307)
Region Coverage: 27.66% (40158/145200)
Branch Coverage: 24.27% (20405/84086)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4e88974b17a72cd2c3476e25a5d2830cd7c5b50c_4e88974b17a72cd2c3476e25a5d2830cd7c5b50c/report/index.html

doris-robot · 2024-09-12T06:21:10Z

TPC-DS: Total hot run time: 199116 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4e88974b17a72cd2c3476e25a5d2830cd7c5b50c, data reload: false
query1	1259	889	889	889
query2	6080	1741	1699	1699
query3	10641	3793	3916	3793
query4	54660	25809	23699	23699
query5	5019	552	526	526
query6	381	176	169	169
query7	5673	319	304	304
query8	305	226	230	226
query9	7053	2425	2425	2425
query10	401	286	276	276
query11	15971	15425	15809	15425
query12	159	100	100	100
query13	1487	390	379	379
query14	10250	7069	7163	7069
query15	206	187	184	184
query16	7068	493	495	493
query17	1149	600	619	600
query18	1798	314	318	314
query19	214	157	156	156
query20	122	114	124	114
query21	208	106	105	105
query22	4821	4615	4653	4615
query23	34739	34133	33677	33677
query24	6148	3063	3053	3053
query25	527	411	413	411
query26	703	160	158	158
query27	1739	287	286	286
query28	2789	2024	1997	1997
query29	679	437	438	437
query30	231	159	158	158
query31	971	761	807	761
query32	76	55	55	55
query33	480	307	308	307
query34	902	483	481	481
query35	857	756	743	743
query36	1056	874	918	874
query37	141	85	87	85
query38	4013	3947	3990	3947
query39	1453	1401	1407	1401
query40	211	119	116	116
query41	49	48	47	47
query42	121	101	98	98
query43	481	439	441	439
query44	1177	758	753	753
query45	201	174	176	174
query46	1102	813	810	810
query47	1910	1800	1808	1800
query48	364	295	294	294
query49	774	464	452	452
query50	955	456	542	456
query51	6999	6947	6904	6904
query52	105	87	88	87
query53	257	184	178	178
query54	558	452	456	452
query55	76	75	77	75
query56	291	265	253	253
query57	1203	1071	1127	1071
query58	227	234	244	234
query59	2644	2488	2380	2380
query60	293	282	276	276
query61	115	105	101	101
query62	806	672	670	670
query63	226	183	179	179
query64	1688	730	683	683
query65	3260	3159	3204	3159
query66	685	339	340	339
query67	16209	15648	15499	15499
query68	2006	876	855	855
query69	444	334	330	330
query70	1158	1146	1136	1136
query71	365	352	345	345
query72	5961	3373	3431	3373
query73	596	577	592	577
query74	9295	9090	9043	9043
query75	3109	3022	3037	3022
query76	994	877	857	857
query77	519	411	407	407
query78	9545	9323	9490	9323
query79	933	890	875	875
query80	893	857	818	818
query81	458	274	264	264
query82	276	269	267	267
query83	194	195	195	195
query84	262	108	110	108
query85	651	417	409	409
query86	339	321	324	321
query87	4391	4343	4419	4343
query88	4156	4036	4065	4036
query89	380	370	380	370
query90	1738	317	312	312
query91	127	124	123	123
query92	79	79	80	79
query93	1074	1065	1056	1056
query94	645	380	350	350
query95	444	484	421	421
query96	473	477	474	474
query97	3134	3124	3146	3124
query98	240	226	236	226
query99	1662	1307	1308	1307
Total cold run time: 291957 ms
Total hot run time: 199116 ms

doris-robot · 2024-09-12T06:26:41Z

ClickBench: Total hot run time: 30.93 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4e88974b17a72cd2c3476e25a5d2830cd7c5b50c, data reload: false
query1	0.05	0.05	0.04
query2	0.08	0.04	0.05
query3	0.22	0.06	0.05
query4	1.67	0.06	0.06
query5	0.51	0.49	0.50
query6	1.14	0.72	0.72
query7	0.01	0.01	0.01
query8	0.05	0.04	0.05
query9	0.56	0.51	0.51
query10	0.58	0.59	0.56
query11	0.16	0.12	0.12
query12	0.15	0.13	0.12
query13	0.62	0.61	0.62
query14	1.47	1.49	1.50
query15	0.90	0.88	0.87
query16	0.36	0.37	0.38
query17	1.07	1.03	1.02
query18	0.22	0.21	0.21
query19	1.90	1.78	1.83
query20	0.01	0.01	0.01
query21	15.41	0.68	0.68
query22	4.08	8.39	1.23
query23	17.93	1.33	1.29
query24	2.29	0.21	0.21
query25	0.19	0.09	0.07
query26	0.29	0.18	0.17
query27	0.08	0.09	0.08
query28	13.23	1.01	0.98
query29	12.55	3.38	3.34
query30	0.24	0.05	0.07
query31	2.86	0.42	0.41
query32	3.24	0.49	0.49
query33	3.02	3.10	3.04
query34	15.45	4.35	4.32
query35	4.31	4.34	4.40
query36	0.68	0.49	0.48
query37	0.18	0.16	0.16
query38	0.17	0.16	0.16
query39	0.04	0.05	0.04
query40	0.17	0.13	0.14
query41	0.11	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.05	0.04
Total cold run time: 108.36 s
Total hot run time: 30.93 s

github-actions · 2024-09-12T08:46:14Z

PR approved by at least one committer and no changes requested.

deardeng

LGTM

yujun777

LGTM

github-actions · 2024-09-12T09:18:02Z

PR approved by anyone and no changes requested.

1. add a config deploy_mode to enable disaggregated mode. In this mode users have to config cloud_instance_id and meta_service_endpoint. 2. When a fe starts from empty, it would try to create instance. 3. If the instance does not exists, it creates and starts from master role. If the instance exists, then it stats with its role got from ms. 4. Frontends are added via sql alter system add frontend. Backends are added via sql alter system add backend. 5. Users do not need config cloud_instance_id and meta_service_endpoint in be.conf, because fe sends them to be via heartbeat. 6. Builtin vault is not needed any more, internal tabels are stored in default vault. TODO: 1. decomission fe and be via sql 2. change cloud_instance_id to cluster id. doc pr apache/doris-website#1072

…40264) ## Proposed changes 1. add a config deploy_mode to enable disaggregated mode. In this mode users have to config cloud_instance_id and meta_service_endpoint. 2. When a fe starts from empty, it would try to create instance. 3. If the instance does not exists, it creates and starts from master role. If the instance exists, then it stats with its role got from ms. 4. Frontends are added via sql alter system add frontend. Backends are added via sql alter system add backend. 5. Users do not need config cloud_instance_id and meta_service_endpoint in be.conf, because fe sends them to be via heartbeat. 6. Builtin vault is not needed any more, internal tabels are stored in default vault. TODO: 1. decomission fe and be via sql 2. change cloud_instance_id to cluster id. doc pr apache/doris-website#1072

dataroaring requested review from CalvinKirs and morningman as code owners September 2, 2024 09:35

github-actionsBot reviewed Sep 2, 2024
View reviewed changes

apache deleted a comment from github-actionsBotSep 2, 2024

dataroaring force-pushed the cloud-node-mgr branch from bdf9d4a to 68e9c9dCompare September 2, 2024 13:20

github-actionsBot reviewed Sep 2, 2024
View reviewed changes

Comment threadbe/src/cloud/config.h Outdated

dataroaring force-pushed the cloud-node-mgr branch 2 times, most recently from 2e4f889 to f43e7e7Compare September 2, 2024 13:51

dataroaring force-pushed the cloud-node-mgr branch from f43e7e7 to 31fc019Compare September 3, 2024 11:55

gavinchou requested changes Sep 5, 2024
View reviewed changes

gavinchou reviewed Sep 5, 2024
View reviewed changes

Comment threadfe/fe-core/src/main/java/org/apache/doris/cloud/system/CloudSystemInfoService.java Outdated

dataroaring force-pushed the cloud-node-mgr branch from 31fc019 to 7cd8aaaCompare September 8, 2024 18:01

deardeng reviewed Sep 9, 2024
View reviewed changes

Comment threadbe/src/agent/heartbeat_server.cpp Outdated

dataroaring force-pushed the cloud-node-mgr branch from 5767a7d to 3a1bfb2Compare September 9, 2024 15:49

dataroaring added 2 commits September 12, 2024 13:38

fix
b8c4c8e

fix
4e88974

gavinchou approved these changes Sep 12, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 12, 2024

CalvinKirs approved these changes Sep 12, 2024
View reviewed changes

deardeng approved these changes Sep 12, 2024
View reviewed changes

yujun777 approved these changes Sep 12, 2024
View reviewed changes

github-actionsBot added the reviewed label Sep 12, 2024

dataroaring merged commit d6132bb into apache:masterSep 12, 2024

gavinchou added dev/3.0.2-merged dev/3.0.x dev/3.0.x-conflict and removed dev/3.0.x dev/3.0.2-merged labels Sep 12, 2024

dataroaring added dev/3.0.2-merged and removed dev/3.0.x-conflict labels Sep 13, 2024

gavinchou mentioned this pull request Oct 13, 2024
Release Note 3.0.2 #41558
Closed

Uh oh!

Conversation

dataroaring commented Sep 2, 2024• edited by gavinchou Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Uh oh!

doris-robot commented Sep 2, 2024

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dataroaring commented Sep 2, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dataroaring commented Sep 9, 2024

Uh oh!

dataroaring commented Sep 9, 2024

Uh oh!

Uh oh!

dataroaring commented Sep 9, 2024

Uh oh!

doris-robot commented Sep 9, 2024

Uh oh!

dataroaring commented Sep 9, 2024

Uh oh!

doris-robot commented Sep 9, 2024

Uh oh!

doris-robot commented Sep 9, 2024

Uh oh!

doris-robot commented Sep 9, 2024

Uh oh!

doris-robot commented Sep 9, 2024

Uh oh!

dataroaring commented Sep 9, 2024

Uh oh!

doris-robot commented Sep 9, 2024

Uh oh!

doris-robot commented Sep 9, 2024

Uh oh!

doris-robot commented Sep 9, 2024

Uh oh!

doris-robot commented Sep 9, 2024

Uh oh!

doris-robot commented Sep 11, 2024

Uh oh!

doris-robot commented Sep 11, 2024

Uh oh!

doris-robot commented Sep 11, 2024

Uh oh!

dataroaring commented Sep 12, 2024

Uh oh!

doris-robot commented Sep 12, 2024

Uh oh!

doris-robot commented Sep 12, 2024

Uh oh!

doris-robot commented Sep 12, 2024

Uh oh!

doris-robot commented Sep 12, 2024

Uh oh!

github-actionsBot commented Sep 12, 2024

Uh oh!

deardeng left a comment

Choose a reason for hiding this comment

Uh oh!

yujun777 left a comment

Choose a reason for hiding this comment

dataroaring commented Sep 2, 2024•
edited by gavinchou
Loading