Skip to content

[Fix](executor)Fix normal group can not be appended when image exits - #33197

Merged
yiguolei merged 1 commit into
apache:masterfrom
wangbo:0403_fix_set_normal
Apr 3, 2024
Merged

[Fix](executor)Fix normal group can not be appended when image exits#33197
yiguolei merged 1 commit into
apache:masterfrom
wangbo:0403_fix_set_normal

Conversation

@wangbo

@wangbowangbo commented Apr 3, 2024

Copy link
Copy Markdown
Contributor

Proposed changes

Fix normal group can not be appended when image exits.
Currently when FE images exits,then normal group could be put into WorkloadGroupMgr failed because image's WorkloadGroup meta can overwrite normal group.

@doris-robot

doris-robot commented Apr 3, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

yiguolei
yiguolei reviewed Apr 3, 2024
View reviewed changes
Comment threadfe/fe-core/src/main/java/org/apache/doris/catalog/Env.java Outdated
yiguolei
yiguolei reviewed Apr 3, 2024
View reviewed changes
Comment threadfe/fe-core/src/main/java/org/apache/doris/resource/workloadgroup/WorkloadGroupMgr.java Outdated
@wangbo
wangboforce-pushed the 0403_fix_set_normal branch 2 times, most recently from cfe3f71 to 3dc2208CompareApril 3, 2024 06:48
@wangbo
wangboforce-pushed the 0403_fix_set_normal branch from 3dc2208 to acfbf39CompareApril 3, 2024 06:50
@wangbo

wangbo commented Apr 3, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@yiguoleiyiguolei added the usercase Important user case type label label Apr 3, 2024
@github-actions

github-actionsBot commented Apr 3, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actionsgithub-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Apr 3, 2024
@github-actions

github-actionsBot commented Apr 3, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@doris-robot

doris-robot commented Apr 3, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 38538 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit acfbf3922e532fb22144223acfaefa4e572e2da6, data reload: false
------ Round 1 ----------------------------------
q1	18297	4095	4078	4078
q2	2024	201	180	180
q3	10478	1203	1326	1203
q4	10529	862	994	862
q5	7516	2925	2934	2925
q6	220	135	135	135
q7	1107	630	595	595
q8	9414	1968	2044	1968
q9	6686	6180	6165	6165
q10	8412	3515	3509	3509
q11	417	240	236	236
q12	371	211	204	204
q13	17773	2895	2899	2895
q14	266	241	234	234
q15	519	479	477	477
q16	480	377	364	364
q17	953	918	928	918
q18	7305	6578	6356	6356
q19	1578	1549	1528	1528
q20	558	328	311	311
q21	3467	3096	3107	3096
q22	349	305	299	299
Total cold run time: 108719 ms
Total hot run time: 38538 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4042	4052	4037	4037
q2	325	222	224	222
q3	2983	2943	2960	2943
q4	1876	1824	1830	1824
q5	5239	5195	5241	5195
q6	208	126	127	126
q7	2275	1797	1798	1797
q8	3182	3285	3275	3275
q9	8498	8484	8427	8427
q10	3755	3821	3837	3821
q11	536	454	439	439
q12	722	535	540	535
q13	16803	2898	2897	2897
q14	286	254	276	254
q15	508	460	466	460
q16	450	404	404	404
q17	1705	1681	1683	1681
q18	7590	7271	7148	7148
q19	1637	1631	1636	1631
q20	1920	1735	1735	1735
q21	5003	4819	4755	4755
q22	485	407	422	407
Total cold run time: 70028 ms
Total hot run time: 54013 ms

@doris-robot

doris-robot commented Apr 3, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 180044 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit acfbf3922e532fb22144223acfaefa4e572e2da6, data reload: false
query1	1218	383	1136	383
query2	6474	1938	1804	1804
query3	6674	209	225	209
query4	24111	21360	21393	21360
query5	4216	397	400	397
query6	263	191	184	184
query7	4605	296	294	294
query8	233	186	177	177
query9	8462	2255	2254	2254
query10	578	248	245	245
query11	14912	14403	14542	14403
query12	142	96	90	90
query13	1634	376	368	368
query14	8686	6864	6721	6721
query15	208	170	175	170
query16	7149	262	263	262
query17	1024	580	578	578
query18	1911	286	288	286
query19	206	158	158	158
query20	98	89	88	88
query21	198	129	127	127
query22	4958	4759	4810	4759
query23	33632	32843	32967	32843
query24	10820	3110	3132	3110
query25	693	396	407	396
query26	1882	158	155	155
query27	3025	328	320	320
query28	6748	1871	1849	1849
query29	1351	587	600	587
query30	323	172	163	163
query31	953	716	713	713
query32	98	54	53	53
query33	652	256	248	248
query34	930	475	480	475
query35	813	683	698	683
query36	983	872	848	848
query37	285	71	71	71
query38	3505	3381	3383	3381
query39	1587	1653	1525	1525
query40	291	128	125	125
query41	49	50	52	50
query42	107	100	101	100
query43	457	424	429	424
query44	1137	702	709	702
query45	270	263	260	260
query46	1070	814	736	736
query47	1891	1794	1784	1784
query48	370	310	302	302
query49	1155	362	368	362
query50	802	382	389	382
query51	6774	6666	6561	6561
query52	106	88	92	88
query53	353	288	281	281
query54	294	245	232	232
query55	87	73	76	73
query56	247	229	223	223
query57	1254	1120	1134	1120
query58	247	213	231	213
query59	2490	2344	2418	2344
query60	255	244	241	241
query61	119	115	112	112
query62	708	455	442	442
query63	305	278	277	277
query64	6476	3184	3226	3184
query65	3078	3019	2990	2990
query66	1446	325	324	324
query67	15668	14983	15299	14983
query68	8226	581	573	573
query69	548	321	300	300
query70	1303	1126	1139	1126
query71	498	267	277	267
query72	6341	2561	2407	2407
query73	817	315	313	313
query74	6636	6323	6420	6323
query75	3618	2296	2281	2281
query76	5796	1178	1222	1178
query77	681	252	244	244
query78	10896	10121	10086	10086
query79	9572	525	523	523
query80	1288	414	412	412
query81	498	238	238	238
query82	345	99	94	94
query83	214	164	165	164
query84	264	87	87	87
query85	1029	283	270	270
query86	375	293	306	293
query87	3672	3499	3472	3472
query88	3744	2263	2264	2263
query89	529	369	357	357
query90	2127	173	171	171
query91	132	117	103	103
query92	56	51	48	48
query93	6559	544	537	537
query94	1274	180	183	180
query95	424	306	305	305
query96	615	269	265	265
query97	2688	2506	2458	2458
query98	224	217	217	217
query99	1186	875	837	837
Total cold run time: 295921 ms
Total hot run time: 180044 ms

@doris-robot

doris-robot commented Apr 3, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 29.3 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit acfbf3922e532fb22144223acfaefa4e572e2da6, data reload: false
query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.24	0.06	0.06
query4	1.67	0.06	0.06
query5	0.50	0.48	0.50
query6	1.14	0.65	0.65
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.56	0.52	0.50
query10	0.56	0.57	0.56
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.61	0.61	0.59
query14	0.77	0.78	0.80
query15	0.85	0.84	0.84
query16	0.35	0.36	0.36
query17	0.97	1.03	1.00
query18	0.25	0.26	0.24
query19	1.80	1.69	1.68
query20	0.02	0.01	0.01
query21	15.45	0.64	0.63
query22	4.02	8.67	1.12
query23	17.75	1.16	1.18
query24	1.59	0.20	0.19
query25	0.16	0.07	0.08
query26	0.27	0.16	0.15
query27	0.08	0.08	0.07
query28	13.78	0.96	0.95
query29	12.54	3.30	3.30
query30	0.26	0.06	0.07
query31	2.84	0.38	0.39
query32	3.29	0.47	0.47
query33	2.91	2.87	2.82
query34	15.51	4.35	4.33
query35	4.38	4.39	4.39
query36	0.68	0.48	0.47
query37	0.19	0.15	0.16
query38	0.15	0.14	0.14
query39	0.04	0.03	0.04
query40	0.17	0.17	0.13
query41	0.11	0.05	0.05
query42	0.05	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 107.03 s
Total hot run time: 29.3 s

@doris-robot

doris-robot commented Apr 3, 2024

Copy link
Copy Markdown

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit acfbf3922e532fb22144223acfaefa4e572e2da6 with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 16.5 seconds inserted 10000000 Rows, about 606K ops/s

@yiguolei
yiguolei merged commit 6310561 into apache:masterApr 3, 2024
yiguolei pushed a commit that referenced this pull request Apr 3, 2024
@wangbo@Doris-Extras
[Fix](executor)Fix normal group can not be appended when image exits#…
df8e397
…33197
@yiguoleiyiguolei mentioned this pull request Apr 7, 2024
Closed
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@wangbo@HappenLee
[Fix](executor)Fix normal group can not be appended when image exitsa…
f82943c
…pache#33197
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguoleiyiguoleiyiguolei approved these changes
@Gabriel39Gabriel39Gabriel39 approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.reviewedusercaseImportant user case type labelworkload-group

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@wangbo@doris-robot@yiguolei@Gabriel39