Skip to content

[bugfix](stop) should skip the loop when graceful stop - #33212

Merged
yiguolei merged 1 commit into
apache:masterfrom
yiguolei:fix_grace_stop
Apr 3, 2024
Merged

[bugfix](stop) should skip the loop when graceful stop#33212
yiguolei merged 1 commit into
apache:masterfrom
yiguolei:fix_grace_stop

Conversation

@yiguolei

@yiguoleiyiguolei commented Apr 3, 2024

Copy link
Copy Markdown
Contributor

Proposed changes

When call graceful stop, BE will wait all background thread stop.
_update_replica_infos_callback() will try to report to FE during this period. It will take a long time.

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot

doris-robot commented Apr 3, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@yiguolei

yiguolei commented Apr 3, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Apr 3, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot

doris-robot commented Apr 3, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 35.65% (8881/24909)
Line Coverage: 27.38% (72909/266306)
Region Coverage: 26.55% (37703/142006)
Branch Coverage: 23.35% (19219/82304)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6c0148b6f5912fca9531a281bb8f997a1214428f_6c0148b6f5912fca9531a281bb8f997a1214428f/report/index.html

@doris-robot

doris-robot commented Apr 3, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 38917 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6c0148b6f5912fca9531a281bb8f997a1214428f, data reload: false
------ Round 1 ----------------------------------
q1	17653	4130	4100	4100
q2	2015	191	186	186
q3	10481	1240	1418	1240
q4	10266	870	1015	870
q5	7503	3029	2955	2955
q6	218	133	132	132
q7	1110	641	601	601
q8	9421	2056	2059	2056
q9	6671	6233	6167	6167
q10	8425	3546	3510	3510
q11	419	257	239	239
q12	383	220	213	213
q13	17765	2914	2929	2914
q14	278	243	246	243
q15	531	486	484	484
q16	493	402	385	385
q17	964	927	921	921
q18	7450	6524	6439	6439
q19	1593	1541	1537	1537
q20	603	315	309	309
q21	3553	3144	3116	3116
q22	369	300	318	300
Total cold run time: 108164 ms
Total hot run time: 38917 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4049	4041	4036	4036
q2	331	217	223	217
q3	2968	2982	2941	2941
q4	1880	1863	1850	1850
q5	5273	5226	5245	5226
q6	212	127	126	126
q7	2248	1805	1819	1805
q8	3222	3305	3311	3305
q9	8516	8520	8500	8500
q10	3750	4356	4024	4024
q11	566	500	463	463
q12	771	670	598	598
q13	15610	3164	3099	3099
q14	305	276	284	276
q15	547	483	485	483
q16	467	444	450	444
q17	1786	1727	1681	1681
q18	8330	7895	7734	7734
q19	1659	1669	1680	1669
q20	2029	1840	1819	1819
q21	5238	5002	5145	5002
q22	514	444	459	444
Total cold run time: 70271 ms
Total hot run time: 55742 ms

@doris-robot

doris-robot commented Apr 3, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 181993 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6c0148b6f5912fca9531a281bb8f997a1214428f, data reload: false
query1	884	1114	1116	1114
query2	6200	1965	1830	1830
query3	6661	206	204	204
query4	23904	21408	21463	21408
query5	4190	403	433	403
query6	281	188	203	188
query7	4595	307	298	298
query8	235	169	181	169
query9	8457	2218	2205	2205
query10	437	265	265	265
query11	15001	14539	14426	14426
query12	148	92	95	92
query13	1627	393	385	385
query14	8475	6889	6819	6819
query15	210	175	173	173
query16	6626	273	269	269
query17	976	598	575	575
query18	1836	292	284	284
query19	211	162	162	162
query20	98	94	97	94
query21	197	130	128	128
query22	4949	4834	4762	4762
query23	33486	32906	32859	32859
query24	12735	3272	3233	3233
query25	733	413	430	413
query26	1998	172	168	168
query27	3432	388	384	384
query28	6922	1946	1900	1900
query29	1373	614	609	609
query30	291	176	176	176
query31	1028	777	777	777
query32	99	67	64	64
query33	741	269	267	267
query34	1098	515	499	499
query35	862	752	733	733
query36	1037	876	898	876
query37	276	83	84	83
query38	3680	3640	3577	3577
query39	1613	1575	1581	1575
query40	240	143	144	143
query41	52	46	48	46
query42	114	104	108	104
query43	461	413	412	412
query44	1163	726	731	726
query45	295	267	282	267
query46	1106	826	780	780
query47	1983	1947	1972	1947
query48	395	334	330	330
query49	966	375	385	375
query50	836	436	417	417
query51	6972	6829	6734	6734
query52	107	96	93	93
query53	358	294	288	288
query54	303	232	232	232
query55	85	76	77	76
query56	244	235	232	232
query57	1263	1153	1137	1137
query58	255	232	235	232
query59	2535	2325	2167	2167
query60	268	233	248	233
query61	89	89	87	87
query62	651	468	444	444
query63	313	283	291	283
query64	6057	3034	3029	3029
query65	3067	3018	2990	2990
query66	1323	313	324	313
query67	15625	15108	14909	14909
query68	9135	554	588	554
query69	576	326	328	326
query70	1429	1120	1118	1118
query71	492	270	288	270
query72	6377	2567	2385	2385
query73	1576	320	321	320
query74	6749	6376	6297	6297
query75	3437	2297	2272	2272
query76	5284	1087	1209	1087
query77	553	238	243	238
query78	10830	10257	10257	10257
query79	9838	532	525	525
query80	1700	424	424	424
query81	527	245	235	235
query82	410	113	107	107
query83	208	165	160	160
query84	273	90	88	88
query85	938	280	272	272
query86	345	293	276	276
query87	3648	3449	3473	3449
query88	3818	2270	2282	2270
query89	543	386	367	367
query90	1998	177	177	177
query91	128	117	105	105
query92	61	50	55	50
query93	6490	529	534	529
query94	1225	193	193	193
query95	437	338	339	338
query96	608	272	270	270
query97	2644	2506	2515	2506
query98	229	224	222	222
query99	1265	834	847	834
Total cold run time: 298499 ms
Total hot run time: 181993 ms

@doris-robot

doris-robot commented Apr 3, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.25 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6c0148b6f5912fca9531a281bb8f997a1214428f, data reload: false
query1	0.04	0.03	0.03
query2	0.07	0.05	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.08
query5	0.50	0.47	0.49
query6	1.13	0.66	0.64
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.50	0.51
query10	0.56	0.57	0.56
query11	0.14	0.11	0.12
query12	0.14	0.12	0.11
query13	0.61	0.60	0.59
query14	0.76	0.79	0.79
query15	0.87	0.85	0.84
query16	0.36	0.36	0.36
query17	1.02	0.99	0.99
query18	0.25	0.23	0.26
query19	1.86	1.73	1.81
query20	0.02	0.01	0.01
query21	15.40	0.75	0.68
query22	2.94	5.77	1.93
query23	17.39	1.22	1.03
query24	1.44	0.21	0.22
query25	0.12	0.08	0.08
query26	0.29	0.17	0.19
query27	0.07	0.08	0.09
query28	13.85	0.96	0.95
query29	12.54	3.39	3.40
query30	0.28	0.08	0.05
query31	2.84	0.38	0.38
query32	3.28	0.47	0.47
query33	2.89	2.82	2.90
query34	15.49	4.34	4.34
query35	4.37	4.38	4.40
query36	0.67	0.48	0.47
query37	0.20	0.17	0.18
query38	0.16	0.16	0.15
query39	0.06	0.04	0.04
query40	0.19	0.16	0.15
query41	0.10	0.05	0.06
query42	0.06	0.06	0.05
query43	0.05	0.04	0.05
Total cold run time: 105.54 s
Total hot run time: 30.25 s

@doris-robot

doris-robot commented Apr 3, 2024

Copy link
Copy Markdown

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 6c0148b6f5912fca9531a281bb8f997a1214428f with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 15.3 seconds inserted 10000000 Rows, about 653K ops/s

@yujun777

yujun777 commented Apr 3, 2024

Copy link
Copy Markdown
Contributor

LGTM

yujun777
yujun777 approved these changes Apr 3, 2024
View reviewed changes

@yujun777yujun777 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

github-actionsBot commented Apr 3, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Apr 3, 2024
@github-actions

github-actionsBot commented Apr 3, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@yiguolei
yiguolei merged commit 6ebf320 into apache:masterApr 3, 2024
yiguolei added a commit that referenced this pull request Apr 3, 2024
@yiguolei@Doris-Extras
[bugfix](stop) should skip the loop when graceful stop (#33212)
fff5c85
Co-authored-by: yiguolei <yiguolei@gmail.com>
@yiguoleiyiguolei mentioned this pull request Apr 7, 2024
Closed
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@yiguolei@Doris-Extras@HappenLee
[bugfix](stop) should skip the loop when graceful stop (apache#33212)
94b75fb
Co-authored-by: yiguolei <yiguolei@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@platonekoplatonekoplatoneko approved these changes
+1 more reviewer
@yujun777yujun777yujun777 approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@yiguolei@doris-robot@yujun777@platoneko@Doris-Extras