[opt](MultiCast) Avoid copying while holding a lock by Mryange · Pull Request #38348 · apache/doris

Mryange · 2024-07-25T03:05:08Z

Proposed changes

pick #37462
The difference is quite large, so it can't be directly picked.

doris-robot · 2024-07-25T03:05:13Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

Mryange · 2024-07-25T03:05:16Z

run buildall

github-actions · 2024-07-25T03:12:45Z

clang-tidy review says "All clean, LGTM! 👍"

Mryange · 2024-07-25T03:13:32Z

run buildall

github-actions · 2024-07-25T03:21:44Z

clang-tidy review says "All clean, LGTM! 👍"

Mryange · 2024-07-25T03:23:24Z

run buildall

github-actions · 2024-07-25T03:30:42Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-07-25T03:43:26Z

TPC-H: Total hot run time: 49663 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d9d837b695993b2df204e1579fd71c999f72f7dd, data reload: false
------ Round 1 ----------------------------------
q1	17935	4396	4339	4339
q2	2072	155	144	144
q3	10441	1878	2130	1878
q4	10370	1237	1318	1237
q5	8522	3889	3925	3889
q6	242	126	124	124
q7	2025	1634	1639	1634
q8	9287	2695	2695	2695
q9	10710	10394	10171	10171
q10	8597	3511	3539	3511
q11	421	245	239	239
q12	468	301	310	301
q13	18345	3975	3985	3975
q14	352	321	341	321
q15	509	465	470	465
q16	693	578	564	564
q17	1132	955	961	955
q18	7364	6853	6867	6853
q19	1789	1616	1602	1602
q20	525	288	298	288
q21	4487	4142	4037	4037
q22	546	446	441	441
Total cold run time: 116832 ms
Total hot run time: 49663 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4323	4305	4286	4286
q2	317	225	223	223
q3	4163	4137	4146	4137
q4	2788	2747	2738	2738
q5	7264	7173	7123	7123
q6	235	120	119	119
q7	3205	2843	2808	2808
q8	4394	4461	4485	4461
q9	16801	16749	16946	16749
q10	4218	4257	4257	4257
q11	785	702	707	702
q12	1033	850	860	850
q13	7107	3758	3719	3719
q14	454	433	440	433
q15	494	458	471	458
q16	740	677	691	677
q17	3797	3790	3913	3790
q18	8838	8790	8830	8790
q19	1717	1722	1665	1665
q20	2383	2152	2077	2077
q21	8438	8514	8532	8514
q22	1042	964	951	951
Total cold run time: 84536 ms
Total hot run time: 79527 ms

doris-robot · 2024-07-25T03:48:11Z

TeamCity be ut coverage result:
Function Coverage: 37.81% (8117/21469)
Line Coverage: 29.47% (66537/225764)
Region Coverage: 28.95% (34296/118470)
Branch Coverage: 24.83% (17619/70962)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d9d837b695993b2df204e1579fd71c999f72f7dd_d9d837b695993b2df204e1579fd71c999f72f7dd/report/index.html

Mryange · 2024-07-25T04:55:16Z

run buildall

github-actions · 2024-07-25T05:02:04Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-07-25T05:16:26Z

TeamCity be ut coverage result:
Function Coverage: 37.80% (8115/21469)
Line Coverage: 29.46% (66522/225767)
Region Coverage: 28.94% (34287/118474)
Branch Coverage: 24.83% (17620/70966)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2d92aba8608022516d158daa1dd9c4edde43768a_2d92aba8608022516d158daa1dd9c4edde43768a/report/index.html

doris-robot · 2024-07-25T05:27:02Z

TPC-H: Total hot run time: 49503 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2d92aba8608022516d158daa1dd9c4edde43768a, data reload: false
------ Round 1 ----------------------------------
q1	17741	4378	4330	4330
q2	2106	155	144	144
q3	10263	1904	1931	1904
q4	10349	1223	1293	1223
q5	8435	3893	3929	3893
q6	235	126	151	126
q7	2052	1617	1600	1600
q8	9295	2720	2698	2698
q9	10436	10237	10130	10130
q10	8608	3538	3457	3457
q11	424	248	245	245
q12	467	310	308	308
q13	18348	4001	3998	3998
q14	341	321	328	321
q15	503	466	456	456
q16	671	569	570	569
q17	1123	980	950	950
q18	7414	6816	6868	6816
q19	1807	1660	1578	1578
q20	543	311	284	284
q21	4405	4133	4026	4026
q22	516	455	447	447
Total cold run time: 116082 ms
Total hot run time: 49503 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4322	4301	4303	4301
q2	317	230	231	230
q3	4189	4158	4143	4143
q4	2774	2744	2721	2721
q5	7165	7100	7083	7083
q6	242	122	121	121
q7	3186	2887	2828	2828
q8	4377	4477	4518	4477
q9	16868	16636	16691	16636
q10	4213	4276	4245	4245
q11	755	690	716	690
q12	1036	849	845	845
q13	7399	3760	3757	3757
q14	444	419	413	413
q15	510	463	462	462
q16	733	695	680	680
q17	3786	3880	3872	3872
q18	8781	8651	8789	8651
q19	1691	1722	1653	1653
q20	2403	2166	2165	2165
q21	8494	8497	8616	8497
q22	1039	1024	947	947
Total cold run time: 84724 ms
Total hot run time: 79417 ms

doris-robot · 2024-07-25T05:39:47Z

TPC-DS: Total hot run time: 205884 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2d92aba8608022516d158daa1dd9c4edde43768a, data reload: false
query1	939	422	387	387
query2	6524	2909	2712	2712
query3	6916	211	209	209
query4	20534	18066	18059	18059
query5	19731	6455	6521	6455
query6	295	224	222	222
query7	4159	303	314	303
query8	430	450	415	415
query9	3175	2715	2649	2649
query10	435	301	304	301
query11	11280	10850	10861	10850
query12	124	81	77	77
query13	5770	718	725	718
query14	18407	13573	13555	13555
query15	369	237	266	237
query16	6625	288	273	273
query17	3543	1614	885	885
query18	2298	430	414	414
query19	207	154	147	147
query20	84	75	84	75
query21	187	100	98	98
query22	5290	5018	5066	5018
query23	33066	32284	32595	32284
query24	8655	6535	6524	6524
query25	534	441	436	436
query26	511	169	164	164
query27	1737	291	295	291
query28	6136	2371	2322	2322
query29	2956	2850	2623	2623
query30	248	165	174	165
query31	920	758	749	749
query32	71	67	61	61
query33	390	253	263	253
query34	849	499	478	478
query35	1113	901	870	870
query36	1216	1067	1259	1067
query37	90	61	60	60
query38	3081	2926	2976	2926
query39	1368	1328	1314	1314
query40	204	95	97	95
query41	47	46	44	44
query42	85	80	82	80
query43	711	725	694	694
query44	1146	720	709	709
query45	245	244	237	237
query46	1219	971	980	971
query47	1892	1823	1747	1747
query48	1032	749	733	733
query49	628	360	392	360
query50	856	636	605	605
query51	4809	4598	4650	4598
query52	101	87	91	87
query53	461	340	328	328
query54	2674	2442	2458	2442
query55	87	82	83	82
query56	242	225	212	212
query57	1104	1093	1051	1051
query58	227	210	199	199
query59	4152	4084	3980	3980
query60	216	195	214	195
query61	98	95	95	95
query62	804	519	567	519
query63	482	342	342	342
query64	2506	1546	1499	1499
query65	3614	3531	3551	3531
query66	780	373	387	373
query67	16629	16989	15069	15069
query68	8412	692	652	652
query69	563	355	378	355
query70	1552	1358	1523	1358
query71	410	304	313	304
query72	6588	3465	3513	3465
query73	737	328	324	324
query74	6370	5870	5934	5870
query75	4522	3818	3645	3645
query76	4733	1142	1197	1142
query77	661	251	257	251
query78	12447	13209	13620	13209
query79	9711	654	671	654
query80	925	413	408	408
query81	508	243	237	237
query82	265	103	99	99
query83	193	133	132	132
query84	260	70	74	70
query85	871	334	332	332
query86	342	308	292	292
query87	3304	3033	3080	3033
query88	5336	2362	2361	2361
query89	356	295	303	295
query90	1973	203	200	200
query91	175	137	157	137
query92	58	57	52	52
query93	966	545	571	545
query94	893	215	213	213
query95	1111	1048	1086	1048
query96	638	331	330	330
query97	6473	6279	6304	6279
query98	189	171	178	171
query99	2865	898	900	898
Total cold run time: 311721 ms
Total hot run time: 205884 ms

doris-robot · 2024-07-25T05:45:15Z

ClickBench: Total hot run time: 31.24 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2d92aba8608022516d158daa1dd9c4edde43768a, data reload: false
query1	0.02	0.02	0.03
query2	0.07	0.03	0.02
query3	0.25	0.04	0.04
query4	1.79	0.07	0.06
query5	0.53	0.52	0.51
query6	1.29	0.61	0.65
query7	0.02	0.01	0.01
query8	0.03	0.02	0.03
query9	0.52	0.47	0.47
query10	0.54	0.53	0.54
query11	0.12	0.09	0.08
query12	0.12	0.09	0.09
query13	0.61	0.62	0.61
query14	0.78	0.78	0.77
query15	0.78	0.76	0.77
query16	0.36	0.35	0.36
query17	1.00	1.02	1.02
query18	0.23	0.26	0.26
query19	1.92	1.84	1.84
query20	0.01	0.01	0.01
query21	15.45	0.54	0.53
query22	2.48	1.98	2.06
query23	17.43	1.00	0.94
query24	5.86	1.68	1.21
query25	0.38	0.09	0.06
query26	0.69	0.15	0.14
query27	0.03	0.04	0.04
query28	6.72	0.74	0.71
query29	12.64	2.26	2.26
query30	0.57	0.55	0.49
query31	2.82	0.39	0.37
query32	3.37	0.49	0.50
query33	3.08	3.12	3.07
query34	15.26	4.81	4.79
query35	4.86	4.84	4.82
query36	1.09	1.01	1.02
query37	0.06	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.02	0.01
query40	0.17	0.14	0.14
query41	0.06	0.01	0.01
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 104.11 s
Total hot run time: 31.24 s

doris-robot · 2024-07-25T05:48:31Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 2d92aba8608022516d158daa1dd9c4edde43768a with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select: 21.8 seconds inserted 10000000 Rows, about 458K ops/s

HappenLee

LGTM

github-actions · 2024-07-29T02:26:24Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-07-29T02:26:26Z

PR approved by anyone and no changes requested.

pick apache#37462 The difference is quite large, so it can't be directly picked.

github-actionsBot added the area/pipeline label Jul 25, 2024

Mryange force-pushed the branch-2.0 branch from 69da0ca to e04ab9eCompare July 25, 2024 03:13

upd
2d92aba

Mryange force-pushed the branch-2.0 branch from d9d837b to 2d92abaCompare July 25, 2024 04:55

HappenLee approved these changes Jul 29, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jul 29, 2024

github-actionsBot added the reviewed label Jul 29, 2024

HappenLee merged commit 535ae2c into apache:branch-2.0Jul 29, 2024

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
[opt](MultiCast) Avoid copying while holding a lock (apache#38348)
dfe3093
pick apache#37462 The difference is quite large, so it can't be directly picked.

Uh oh!

Conversation

Mryange commented Jul 25, 2024

Proposed changes

Uh oh!

doris-robot commented Jul 25, 2024

Uh oh!

Mryange commented Jul 25, 2024

Uh oh!

github-actionsBot commented Jul 25, 2024

Uh oh!

Mryange commented Jul 25, 2024

Uh oh!

github-actionsBot commented Jul 25, 2024

Uh oh!

Mryange commented Jul 25, 2024

Uh oh!

github-actionsBot commented Jul 25, 2024

Uh oh!

doris-robot commented Jul 25, 2024

Uh oh!

doris-robot commented Jul 25, 2024

Uh oh!

Mryange commented Jul 25, 2024

Uh oh!

github-actionsBot commented Jul 25, 2024

Uh oh!

doris-robot commented Jul 25, 2024

Uh oh!

doris-robot commented Jul 25, 2024

Uh oh!

doris-robot commented Jul 25, 2024

Uh oh!

doris-robot commented Jul 25, 2024

Uh oh!

doris-robot commented Jul 25, 2024

Uh oh!

HappenLee left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jul 29, 2024

Uh oh!

github-actionsBot commented Jul 29, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants