Skip to content

[fix] (compaction) fix time series (#38791) - #39053

Merged
airborne12 merged 1 commit into
apache:branch-2.0from
csun5285:time_time_series_2.0
Aug 9, 2024
Merged

[fix] (compaction) fix time series (#38791)#39053
airborne12 merged 1 commit into
apache:branch-2.0from
csun5285:time_time_series_2.0

Conversation

@csun5285

@csun5285csun5285 commented Aug 7, 2024

Copy link
Copy Markdown
Contributor

Proposed changes

pick from master #38791

Issue Number: close #xxx

@csun5285
add log and fix time series
223f4d6
fix
fix build and comment
fix rebase
@doris-robot

doris-robot commented Aug 7, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@csun5285

csun5285 commented Aug 7, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Aug 7, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot

doris-robot commented Aug 7, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 49660 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 223f4d695ee005541c92fdfbe074c23a2964bd2a, data reload: false
------ Round 1 ----------------------------------
q1	18067	4393	4330	4330
q2	2083	154	146	146
q3	10421	1866	1944	1866
q4	10299	1233	1302	1233
q5	8455	3970	3884	3884
q6	237	123	124	123
q7	2057	1595	1607	1595
q8	9372	2726	2697	2697
q9	10462	10445	10133	10133
q10	8648	3480	3492	3480
q11	410	252	240	240
q12	482	305	307	305
q13	18364	4028	4045	4028
q14	354	323	337	323
q15	522	469	475	469
q16	674	585	566	566
q17	1126	977	962	962
q18	7224	6929	6925	6925
q19	1801	1640	1603	1603
q20	537	292	306	292
q21	4391	4103	4019	4019
q22	528	441	449	441
Total cold run time: 116514 ms
Total hot run time: 49660 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4314	4313	4304	4304
q2	321	228	223	223
q3	4146	4146	4127	4127
q4	2739	2758	2742	2742
q5	7195	7136	7156	7136
q6	236	116	117	116
q7	3261	2819	2849	2819
q8	4401	4479	4506	4479
q9	16874	16866	16786	16786
q10	4259	4231	4233	4231
q11	777	698	689	689
q12	1002	856	887	856
q13	7084	3751	3769	3751
q14	446	423	426	423
q15	494	448	453	448
q16	732	684	671	671
q17	3873	3879	3830	3830
q18	8846	8769	8684	8684
q19	1722	1698	1644	1644
q20	2395	2129	2133	2129
q21	8576	8470	8416	8416
q22	1065	986	1042	986
Total cold run time: 84758 ms
Total hot run time: 79490 ms

@doris-robot

doris-robot commented Aug 7, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 37.73% (8100/21467)
Line Coverage: 29.37% (66332/225875)
Region Coverage: 28.87% (34231/118555)
Branch Coverage: 24.76% (17593/71052)
Coverage Report: http://coverage.selectdb-in.cc/coverage/223f4d695ee005541c92fdfbe074c23a2964bd2a_223f4d695ee005541c92fdfbe074c23a2964bd2a/report/index.html

@doris-robot

doris-robot commented Aug 7, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 203048 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 223f4d695ee005541c92fdfbe074c23a2964bd2a, data reload: false
query1	943	391	414	391
query2	6537	2861	2679	2679
query3	6918	209	199	199
query4	20180	17925	17967	17925
query5	19762	6493	6508	6493
query6	291	220	233	220
query7	4152	299	309	299
query8	442	442	464	442
query9	3111	2660	2604	2604
query10	412	300	302	300
query11	11428	10701	10730	10701
query12	124	84	71	71
query13	5620	701	677	677
query14	18291	13322	13639	13322
query15	362	238	261	238
query16	6472	285	261	261
query17	1707	1469	881	881
query18	2314	421	407	407
query19	202	153	155	153
query20	79	78	80	78
query21	191	98	92	92
query22	5174	5111	5035	5035
query23	32591	31780	32089	31780
query24	7258	6624	6474	6474
query25	521	438	418	418
query26	662	168	166	166
query27	2087	299	297	297
query28	6196	2384	2343	2343
query29	2941	2827	2683	2683
query30	251	168	167	167
query31	901	754	775	754
query32	74	67	60	60
query33	401	257	239	239
query34	852	481	489	481
query35	1122	925	929	925
query36	1405	1328	1262	1262
query37	90	59	68	59
query38	3052	2952	2919	2919
query39	1373	1327	1339	1327
query40	296	96	96	96
query41	55	44	44	44
query42	80	82	81	81
query43	707	670	785	670
query44	1142	728	724	724
query45	247	232	238	232
query46	1226	962	992	962
query47	1955	1743	1621	1621
query48	1026	721	727	721
query49	641	381	373	373
query50	861	609	623	609
query51	4738	4694	4610	4610
query52	87	84	85	84
query53	452	325	329	325
query54	2630	2456	2477	2456
query55	90	84	87	84
query56	233	223	216	216
query57	1130	1148	1161	1148
query58	218	198	192	192
query59	4523	4176	3802	3802
query60	221	203	210	203
query61	100	90	99	90
query62	840	478	440	440
query63	487	343	345	343
query64	2726	1562	1519	1519
query65	3642	3556	3564	3556
query66	821	384	382	382
query67	15320	15554	14798	14798
query68	9912	666	658	658
query69	581	350	359	350
query70	1909	1537	1568	1537
query71	424	312	316	312
query72	6507	3479	3471	3471
query73	730	316	325	316
query74	6245	5802	5921	5802
query75	5300	3682	3695	3682
query76	6133	1168	1218	1168
query77	1064	261	258	258
query78	12641	11581	11529	11529
query79	8144	668	645	645
query80	1365	394	402	394
query81	492	236	241	236
query82	1576	95	100	95
query83	174	140	130	130
query84	266	70	69	69
query85	897	330	333	330
query86	337	301	288	288
query87	3224	3011	3087	3011
query88	4748	2300	2280	2280
query89	403	301	293	293
query90	1960	210	201	201
query91	171	136	150	136
query92	54	54	52	52
query93	6329	580	578	578
query94	707	215	210	210
query95	1110	1056	1052	1052
query96	635	319	321	319
query97	6383	6435	6375	6375
query98	185	179	165	165
query99	2915	903	924	903
Total cold run time: 316504 ms
Total hot run time: 203048 ms

@doris-robot

doris-robot commented Aug 7, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.25 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 223f4d695ee005541c92fdfbe074c23a2964bd2a, data reload: false
query1	0.02	0.02	0.02
query2	0.06	0.03	0.02
query3	0.24	0.04	0.05
query4	1.80	0.07	0.06
query5	0.54	0.51	0.54
query6	1.32	0.61	0.61
query7	0.01	0.02	0.02
query8	0.04	0.02	0.02
query9	0.53	0.48	0.49
query10	0.53	0.54	0.54
query11	0.12	0.08	0.09
query12	0.11	0.09	0.08
query13	0.62	0.61	0.61
query14	0.80	0.78	0.78
query15	0.77	0.75	0.77
query16	0.36	0.39	0.36
query17	1.00	1.02	1.01
query18	0.24	0.27	0.25
query19	1.94	1.82	1.81
query20	0.01	0.02	0.01
query21	15.47	0.56	0.55
query22	2.08	2.47	2.20
query23	17.53	1.07	0.88
query24	4.47	1.05	1.57
query25	0.36	0.12	0.04
query26	0.53	0.16	0.16
query27	0.04	0.05	0.04
query28	8.01	0.73	0.72
query29	12.60	2.29	2.26
query30	0.56	0.55	0.53
query31	2.81	0.39	0.38
query32	3.39	0.49	0.49
query33	3.08	3.05	3.11
query34	15.27	4.79	4.78
query35	4.83	4.80	4.86
query36	1.04	1.01	1.02
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.06	0.01	0.02
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 103.5 s
Total hot run time: 31.25 s

@doris-robot

doris-robot commented Aug 7, 2024

Copy link
Copy Markdown

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 223f4d695ee005541c92fdfbe074c23a2964bd2a with default session variables
Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.1 seconds inserted 10000000 Rows, about 473K ops/s

airborne12
airborne12 approved these changes Aug 9, 2024
View reviewed changes

@airborne12airborne12 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

github-actionsBot commented Aug 9, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actionsgithub-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Aug 9, 2024
@github-actions

github-actionsBot commented Aug 9, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@airborne12
airborne12 merged commit f4a98a7 into apache:branch-2.0Aug 9, 2024
GoGoWen pushed a commit to GoGoWen/incubator-doris that referenced this pull request Aug 27, 2024
@csun5285weixingyu12
[fix] (compaction) fix time series (apache#38791) (apache#39053)
194d845
## Proposed changes
pick from master apache#38791
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@airborne12airborne12airborne12 approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@csun5285@doris-robot@airborne12