Skip to content

[fix](ut) repair segcompaction ut - #38165

Merged
dataroaring merged 3 commits into
apache:masterfrom
zhannngchen:fix-segcompaction-ut
Jul 20, 2024
Merged

[fix](ut) repair segcompaction ut#38165
dataroaring merged 3 commits into
apache:masterfrom
zhannngchen:fix-segcompaction-ut

Conversation

@zhannngchen

@zhannngchenzhannngchen commented Jul 20, 2024

Copy link
Copy Markdown
Contributor

Proposed changes

Issue Number: close #xxx

SegcompactionTest is disabled by #16068 due to some code clean up, it needs to be repaired

@doris-robot

doris-robot commented Jul 20, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zhannngchen

zhannngchen commented Jul 20, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@zhannngchen
zhannngchenforce-pushed the fix-segcompaction-ut branch from 846df8b to e0d4e37CompareJuly 20, 2024 03:18
@zhannngchen

zhannngchen commented Jul 20, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@zhannngchen

zhannngchen commented Jul 20, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

github-actions[bot]
github-actionsBot reviewed Jul 20, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/test/olap/segcompaction_test.cpp Outdated
@doris-robot

doris-robot commented Jul 20, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 40158 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e5f05223edd0b7139bc528490e844e129c95d2d5, data reload: false
------ Round 1 ----------------------------------
q1	17635	4449	4427	4427
q2	2017	198	189	189
q3	10431	1242	1091	1091
q4	10199	853	852	852
q5	7559	2692	2668	2668
q6	217	136	136	136
q7	964	593	600	593
q8	9224	2117	2047	2047
q9	8877	6601	6614	6601
q10	8922	3858	3805	3805
q11	452	237	243	237
q12	432	224	220	220
q13	18727	3015	2979	2979
q14	283	243	250	243
q15	532	488	483	483
q16	499	387	374	374
q17	988	608	714	608
q18	8292	7609	7334	7334
q19	7835	1474	1475	1474
q20	667	316	323	316
q21	4992	3200	3922	3200
q22	344	281	281	281
Total cold run time: 120088 ms
Total hot run time: 40158 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4376	4260	4228	4228
q2	377	284	270	270
q3	3217	2942	2912	2912
q4	1995	1804	1711	1711
q5	5538	5519	5474	5474
q6	230	141	137	137
q7	2293	1849	1839	1839
q8	3288	3392	3399	3392
q9	8770	8902	8796	8796
q10	4071	3802	3883	3802
q11	608	500	496	496
q12	814	612	619	612
q13	17255	3142	3197	3142
q14	337	285	295	285
q15	556	493	496	493
q16	514	441	457	441
q17	1855	1530	1524	1524
q18	8483	8081	7896	7896
q19	1732	1391	1561	1391
q20	2322	1886	1856	1856
q21	5177	4861	5087	4861
q22	598	524	509	509
Total cold run time: 74406 ms
Total hot run time: 56067 ms

@doris-robot

doris-robot commented Jul 20, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 173129 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e5f05223edd0b7139bc528490e844e129c95d2d5, data reload: false
query1	922	374	362	362
query2	6350	1878	1791	1791
query3	6628	212	220	212
query4	25898	17444	17373	17373
query5	3698	473	476	473
query6	261	184	170	170
query7	4590	289	284	284
query8	253	194	201	194
query9	8531	2451	2425	2425
query10	436	282	273	273
query11	11614	10010	9956	9956
query12	112	82	84	82
query13	1639	380	372	372
query14	10001	7760	7837	7760
query15	220	171	167	167
query16	7591	454	497	454
query17	1561	550	524	524
query18	1689	283	267	267
query19	189	150	177	150
query20	91	81	80	80
query21	208	136	120	120
query22	4293	4054	3980	3980
query23	33982	33660	33618	33618
query24	11370	2912	2830	2830
query25	639	410	402	402
query26	1080	147	149	147
query27	2300	270	280	270
query28	7366	2068	2052	2052
query29	931	635	627	627
query30	268	155	159	155
query31	987	758	765	758
query32	87	57	57	57
query33	778	337	367	337
query34	938	496	493	493
query35	905	758	760	758
query36	1147	966	965	965
query37	125	76	79	76
query38	2891	2807	2743	2743
query39	866	834	792	792
query40	205	121	116	116
query41	49	43	42	42
query42	125	100	102	100
query43	492	468	458	458
query44	1194	725	729	725
query45	188	158	161	158
query46	1095	735	737	735
query47	1901	1767	1768	1767
query48	358	290	291	290
query49	835	401	403	401
query50	775	386	387	386
query51	6972	6732	6750	6732
query52	105	96	93	93
query53	355	280	282	280
query54	929	463	449	449
query55	75	72	73	72
query56	295	260	260	260
query57	1123	1066	1041	1041
query58	263	241	265	241
query59	2725	2657	2583	2583
query60	291	279	265	265
query61	95	91	91	91
query62	791	651	617	617
query63	318	292	285	285
query64	9420	2260	1682	1682
query65	3156	3090	3110	3090
query66	720	324	326	324
query67	15366	14965	14745	14745
query68	4416	533	526	526
query69	517	395	343	343
query70	1154	1062	1186	1062
query71	393	276	273	273
query72	6917	5798	5508	5508
query73	752	329	319	319
query74	6183	5701	5665	5665
query75	3402	2654	2654	2654
query76	2779	892	938	892
query77	477	309	305	305
query78	9988	8869	8900	8869
query79	3543	519	514	514
query80	1472	462	462	462
query81	569	222	215	215
query82	1056	131	134	131
query83	196	167	166	166
query84	270	87	87	87
query85	1422	310	295	295
query86	446	296	293	293
query87	3330	3074	3122	3074
query88	4123	2351	2358	2351
query89	491	372	372	372
query90	1872	195	192	192
query91	127	100	97	97
query92	60	48	49	48
query93	4840	510	498	498
query94	1126	286	294	286
query95	403	327	311	311
query96	613	272	269	269
query97	3171	3021	3025	3021
query98	219	199	202	199
query99	1728	1260	1255	1255
Total cold run time: 281706 ms
Total hot run time: 173129 ms

@doris-robot

doris-robot commented Jul 20, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.4 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e5f05223edd0b7139bc528490e844e129c95d2d5, data reload: false
query1	0.04	0.03	0.04
query2	0.08	0.03	0.04
query3	0.22	0.04	0.05
query4	1.67	0.10	0.07
query5	0.49	0.48	0.48
query6	1.14	0.73	0.71
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.56	0.51	0.48
query10	0.53	0.55	0.55
query11	0.15	0.11	0.11
query12	0.14	0.12	0.13
query13	0.61	0.58	0.59
query14	0.75	0.77	0.78
query15	0.86	0.81	0.82
query16	0.37	0.37	0.36
query17	1.03	1.05	1.01
query18	0.22	0.21	0.22
query19	1.80	1.73	1.75
query20	0.01	0.01	0.01
query21	15.39	0.75	0.67
query22	4.49	6.97	1.79
query23	18.36	1.34	1.24
query24	2.15	0.22	0.22
query25	0.15	0.08	0.08
query26	0.29	0.22	0.22
query27	0.46	0.24	0.23
query28	13.24	1.02	1.00
query29	12.60	3.26	3.29
query30	0.26	0.06	0.05
query31	2.88	0.41	0.38
query32	3.26	0.49	0.47
query33	2.88	2.91	2.91
query34	17.08	4.31	4.29
query35	4.41	4.41	4.38
query36	0.66	0.47	0.46
query37	0.19	0.15	0.16
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.16	0.12	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.04 s
Total hot run time: 30.4 s

@zhannngchen@github-actions
Update be/test/olap/segcompaction_test.cpp
03eb24e
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@zhannngchen

zhannngchen commented Jul 20, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Jul 20, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot

doris-robot commented Jul 20, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 39622 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 03eb24e2f48b7f6edb4d71c71f4b3f869c986999, data reload: false
------ Round 1 ----------------------------------
q1	17633	4256	4218	4218
q2	2013	194	186	186
q3	10438	1156	1103	1103
q4	10177	785	771	771
q5	7568	2674	2717	2674
q6	216	135	139	135
q7	958	596	589	589
q8	9217	2060	2043	2043
q9	8853	6523	6535	6523
q10	8897	3757	3766	3757
q11	449	237	237	237
q12	481	224	216	216
q13	18844	2958	3023	2958
q14	279	224	241	224
q15	524	484	482	482
q16	493	388	390	388
q17	956	625	680	625
q18	7971	7387	7359	7359
q19	6668	1428	1407	1407
q20	675	320	331	320
q21	4854	3130	3144	3130
q22	344	290	277	277
Total cold run time: 118508 ms
Total hot run time: 39622 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4455	4223	4191	4191
q2	368	265	264	264
q3	2986	2863	2957	2863
q4	2011	1688	1770	1688
q5	5613	5494	5464	5464
q6	224	143	130	130
q7	2237	1874	1807	1807
q8	3231	3392	3373	3373
q9	8703	8772	8791	8772
q10	4077	3799	3795	3795
q11	598	503	532	503
q12	798	621	641	621
q13	15845	3221	3156	3156
q14	321	291	298	291
q15	516	493	494	493
q16	504	449	454	449
q17	1828	1510	1446	1446
q18	8035	7765	7720	7720
q19	1765	1664	1686	1664
q20	2476	1889	1858	1858
q21	8270	4901	4688	4688
q22	584	510	532	510
Total cold run time: 75445 ms
Total hot run time: 55746 ms

@doris-robot

doris-robot commented Jul 20, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 174086 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 03eb24e2f48b7f6edb4d71c71f4b3f869c986999, data reload: false
query1	911	388	380	380
query2	6429	1899	1757	1757
query3	6641	210	218	210
query4	28427	17546	17596	17546
query5	3621	472	485	472
query6	275	221	187	187
query7	4575	288	292	288
query8	245	200	195	195
query9	8528	2454	2431	2431
query10	456	280	290	280
query11	11839	10078	9997	9997
query12	120	87	85	85
query13	1646	373	382	373
query14	10322	7845	7194	7194
query15	223	168	172	168
query16	7707	452	462	452
query17	1587	540	536	536
query18	1955	287	283	283
query19	202	148	149	148
query20	93	84	85	84
query21	210	131	128	128
query22	4418	4146	4027	4027
query23	34218	33637	33685	33637
query24	11060	2928	2961	2928
query25	588	382	414	382
query26	1026	158	155	155
query27	2259	282	277	277
query28	6696	2057	2080	2057
query29	940	664	674	664
query30	271	158	161	158
query31	976	764	752	752
query32	94	53	55	53
query33	757	342	339	339
query34	982	501	523	501
query35	868	775	772	772
query36	1166	1007	978	978
query37	139	85	83	83
query38	2943	2934	2835	2835
query39	918	900	856	856
query40	224	119	117	117
query41	44	43	44	43
query42	114	102	93	93
query43	491	459	475	459
query44	1223	730	734	730
query45	191	159	162	159
query46	1095	729	747	729
query47	1863	1764	1762	1762
query48	374	299	289	289
query49	839	407	413	407
query50	776	393	400	393
query51	7024	6902	6808	6808
query52	104	98	92	92
query53	364	295	293	293
query54	906	478	449	449
query55	74	73	77	73
query56	291	273	268	268
query57	1154	1042	1069	1042
query58	255	247	269	247
query59	2744	2745	2466	2466
query60	304	287	288	287
query61	97	96	95	95
query62	820	658	655	655
query63	316	288	287	287
query64	9682	2265	1684	1684
query65	3227	3149	3132	3132
query66	819	329	332	329
query67	15389	15026	14974	14974
query68	4642	554	536	536
query69	641	449	358	358
query70	1177	1082	1057	1057
query71	422	280	280	280
query72	7268	5492	5475	5475
query73	774	324	321	321
query74	6050	5667	5690	5667
query75	3505	2714	2711	2711
query76	2772	960	928	928
query77	633	320	310	310
query78	10544	9232	9098	9098
query79	4482	539	531	531
query80	1227	487	488	487
query81	602	233	238	233
query82	767	144	146	144
query83	209	171	171	171
query84	274	94	88	88
query85	1342	317	301	301
query86	403	323	315	315
query87	3313	3121	3066	3066
query88	4755	2381	2369	2369
query89	509	394	385	385
query90	1823	193	193	193
query91	131	103	102	102
query92	62	48	52	48
query93	5118	510	511	510
query94	954	292	299	292
query95	412	323	377	323
query96	603	271	269	269
query97	3212	3023	3036	3023
query98	223	208	204	204
query99	1599	1281	1307	1281
Total cold run time: 287583 ms
Total hot run time: 174086 ms

@doris-robot

doris-robot commented Jul 20, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.59 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 03eb24e2f48b7f6edb4d71c71f4b3f869c986999, data reload: false
query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.04
query4	1.68	0.07	0.07
query5	0.48	0.48	0.50
query6	1.14	0.72	0.71
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.56	0.49	0.48
query10	0.52	0.56	0.55
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.61	0.59	0.59
query14	0.77	0.81	0.76
query15	0.86	0.81	0.80
query16	0.38	0.34	0.37
query17	0.98	1.04	1.03
query18	0.22	0.22	0.22
query19	1.91	1.74	1.72
query20	0.01	0.01	0.02
query21	15.68	0.77	0.66
query22	4.86	7.10	1.83
query23	18.32	1.34	1.34
query24	2.11	0.23	0.22
query25	0.15	0.10	0.09
query26	0.29	0.21	0.21
query27	0.46	0.23	0.23
query28	13.39	1.01	0.99
query29	12.60	3.28	3.26
query30	0.26	0.06	0.06
query31	2.86	0.39	0.39
query32	3.27	0.47	0.47
query33	2.94	2.95	2.91
query34	17.15	4.38	4.36
query35	4.38	4.38	4.42
query36	0.66	0.46	0.47
query37	0.19	0.15	0.15
query38	0.15	0.15	0.15
query39	0.04	0.04	0.03
query40	0.15	0.13	0.12
query41	0.10	0.04	0.04
query42	0.06	0.05	0.05
query43	0.04	0.03	0.04
Total cold run time: 110.95 s
Total hot run time: 30.59 s

dataroaring
dataroaring approved these changes Jul 20, 2024
View reviewed changes

@dataroaringdataroaring left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

github-actionsBot commented Jul 20, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Jul 20, 2024
@github-actions

github-actionsBot commented Jul 20, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@dataroaring
dataroaring merged commit d1af5d5 into apache:masterJul 20, 2024
dataroaring pushed a commit that referenced this pull request Jul 22, 2024
@zhannngchen@github-actions@dataroaring
[fix](ut) repair segcompaction ut (#38165)
b77441e
SegcompactionTest is disabled by #16068 due to some code clean up, it
needs to be repaired
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
zhannngchen added a commit to zhannngchen/incubator-doris that referenced this pull request Jul 22, 2024
@zhannngchen@github-actions
[fix](ut) repair segcompaction ut (apache#38165)
d92fd7b
SegcompactionTest is disabled by apache#16068 due to some code clean up, it
needs to be repaired
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@zhannngchenzhannngchen mentioned this pull request Jul 22, 2024
Merged
zhannngchen added a commit to zhannngchen/incubator-doris that referenced this pull request Aug 9, 2024
@zhannngchen@github-actions
[fix](ut) repair segcompaction ut (apache#38165)
a29e7a8
SegcompactionTest is disabled by apache#16068 due to some code clean up, it
needs to be repaired
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
zhannngchen added a commit that referenced this pull request Aug 9, 2024
@zhannngchen
[fix](ut) repair segcompaction ut (#38165) (#38225)
8a682d4
cherry-pick #38165
zhannngchen added a commit to zhannngchen/incubator-doris that referenced this pull request Aug 9, 2024
@zhannngchen
[fix](ut) repair segcompaction ut (apache#38165) (apache#38225)
5ff1d5e
cherry-pick apache#38165
@zhannngchenzhannngchen mentioned this pull request Aug 9, 2024
Merged
zhannngchen added a commit to zhannngchen/incubator-doris that referenced this pull request Aug 20, 2024
@zhannngchen
[fix](ut) repair segcompaction ut (apache#38165)
d98cb1e
zhannngchen added a commit that referenced this pull request Aug 21, 2024
@zhannngchen
[fix](ut) repair segcompaction ut (#38165) (#38225) (#39153)
7e52a1a
cherry-pick #38165 and #22928
mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
@zhannngchen@Baymine
[fix](ut) repair segcompaction ut (apache#38165) (apache#38225) (apac…
7f1ae42
…he#39153)
cherry-pick apache#38165 and apache#22928
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@zhannngchen@github-actions@HappenLee
[fix](ut) repair segcompaction ut (apache#38165)
364f998
SegcompactionTest is disabled by apache#16068 due to some code clean up, it
needs to be repaired
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actionsgithub-actions[bot]github-actions[bot] left review comments
@Yukang-LianYukang-LianYukang-Lian approved these changes
@dataroaringdataroaringdataroaring approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.0.15-mergeddev/2.1.6-mergeddev/3.0.1-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@zhannngchen@doris-robot@Yukang-Lian@dataroaring@wm1581066