Skip to content

[Fix](inverted index) fix race condition for column reader load inverted index reader - #34922

Merged
dataroaring merged 1 commit into
apache:masterfrom
airborne12:fix-race
May 16, 2024
Merged

[Fix](inverted index) fix race condition for column reader load inverted index reader#34922
dataroaring merged 1 commit into
apache:masterfrom
airborne12:fix-race

Conversation

@airborne12

@airborne12airborne12 commented May 15, 2024

Copy link
Copy Markdown
Member

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot

doris-robot commented May 15, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@airborne12

airborne12 commented May 15, 2024

Copy link
Copy Markdown
MemberAuthor

run buildall

dataroaring
dataroaring approved these changes May 15, 2024
View reviewed changes

@dataroaringdataroaring left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

github-actionsBot commented May 15, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label May 15, 2024
@github-actions

github-actionsBot commented May 15, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented May 15, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

eldenmoon
eldenmoon approved these changes May 15, 2024
View reviewed changes

@eldenmooneldenmoon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@doris-robot

doris-robot commented May 15, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 187463 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 10335ad424b94551283ad3e5bee51bd97281f3e8, data reload: false
query1	817	393	373	373
query2	6458	2439	2214	2214
query3	6421	224	218	218
query4	23168	21373	21176	21176
query5	3935	415	418	415
query6	260	174	164	164
query7	4412	295	281	281
query8	240	191	198	191
query9	8508	2438	2395	2395
query10	409	253	267	253
query11	14634	14183	14177	14177
query12	139	95	88	88
query13	1580	373	374	373
query14	8802	8055	8511	8055
query15	280	172	172	172
query16	8130	267	260	260
query17	1819	579	553	553
query18	2059	288	275	275
query19	210	161	153	153
query20	93	91	90	90
query21	200	134	131	131
query22	5123	4974	4902	4902
query23	34343	33627	33700	33627
query24	10746	2937	2852	2852
query25	605	373	366	366
query26	1321	159	150	150
query27	2970	317	316	316
query28	7540	2033	2024	2024
query29	868	628	593	593
query30	235	176	182	176
query31	967	776	735	735
query32	88	52	52	52
query33	728	247	238	238
query34	1067	473	477	473
query35	792	702	665	665
query36	1044	931	952	931
query37	146	71	73	71
query38	2907	2813	2767	2767
query39	1630	1556	1569	1556
query40	200	130	120	120
query41	46	44	43	43
query42	106	98	96	96
query43	577	540	535	535
query44	1192	718	747	718
query45	277	253	255	253
query46	1074	703	702	702
query47	1959	1882	1901	1882
query48	383	288	294	288
query49	923	389	394	389
query50	774	371	378	371
query51	6790	6778	6636	6636
query52	111	90	91	90
query53	347	282	285	282
query54	869	431	435	431
query55	74	73	76	73
query56	237	220	220	220
query57	1227	1146	1176	1146
query58	220	197	203	197
query59	3300	3135	3172	3135
query60	253	231	238	231
query61	88	89	87	87
query62	635	496	459	459
query63	311	279	287	279
query64	9911	7386	7380	7380
query65	3197	3104	3140	3104
query66	1361	345	341	341
query67	15550	15260	15108	15108
query68	4592	523	526	523
query69	471	304	299	299
query70	1134	1132	1068	1068
query71	405	262	270	262
query72	7360	2580	2357	2357
query73	705	354	319	319
query74	6504	6099	6238	6099
query75	3332	2614	2612	2612
query76	2814	995	966	966
query77	435	271	267	267
query78	10570	10282	10033	10033
query79	2288	512	512	512
query80	893	429	428	428
query81	532	245	246	245
query82	1271	99	99	99
query83	239	167	170	167
query84	235	88	86	86
query85	1532	272	257	257
query86	471	311	326	311
query87	3333	3124	3102	3102
query88	3851	2303	2313	2303
query89	464	385	381	381
query90	2054	188	239	188
query91	128	97	95	95
query92	57	48	50	48
query93	1960	503	489	489
query94	1245	181	179	179
query95	393	290	298	290
query96	591	273	266	266
query97	3152	3008	3029	3008
query98	243	218	217	217
query99	1224	915	890	890
Total cold run time: 282164 ms
Total hot run time: 187463 ms

@doris-robot

doris-robot commented May 15, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 35.71% (8998/25199)
Line Coverage: 27.38% (74454/271948)
Region Coverage: 26.62% (38490/144611)
Branch Coverage: 23.43% (19635/83802)
Coverage Report: http://coverage.selectdb-in.cc/coverage/10335ad424b94551283ad3e5bee51bd97281f3e8_10335ad424b94551283ad3e5bee51bd97281f3e8/report/index.html

@doris-robot

doris-robot commented May 15, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.24 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 10335ad424b94551283ad3e5bee51bd97281f3e8, data reload: false
query1	0.04	0.03	0.03
query2	0.09	0.04	0.04
query3	0.24	0.06	0.05
query4	1.65	0.09	0.07
query5	0.47	0.49	0.51
query6	1.12	0.71	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.54	0.50	0.49
query10	0.53	0.54	0.54
query11	0.15	0.11	0.11
query12	0.16	0.12	0.12
query13	0.59	0.59	0.60
query14	0.78	0.77	0.76
query15	0.82	0.81	0.82
query16	0.36	0.36	0.37
query17	0.95	0.94	0.99
query18	0.23	0.23	0.25
query19	1.76	1.69	1.69
query20	0.01	0.01	0.02
query21	15.74	0.65	0.64
query22	4.58	7.20	1.48
query23	18.31	1.41	1.39
query24	1.93	0.22	0.22
query25	0.14	0.09	0.10
query26	0.27	0.17	0.18
query27	0.08	0.09	0.07
query28	13.33	1.03	1.00
query29	13.69	3.43	3.38
query30	0.24	0.07	0.05
query31	2.85	0.38	0.38
query32	3.30	0.47	0.48
query33	2.83	2.81	2.84
query34	17.26	4.41	4.43
query35	4.49	4.58	4.57
query36	0.67	0.46	0.46
query37	0.18	0.15	0.15
query38	0.15	0.15	0.14
query39	0.04	0.03	0.03
query40	0.16	0.12	0.14
query41	0.09	0.05	0.05
query42	0.05	0.04	0.05
query43	0.04	0.03	0.04
Total cold run time: 110.98 s
Total hot run time: 30.24 s

@dataroaring

dataroaring commented May 15, 2024

Copy link
Copy Markdown
Contributor

run performance

xiaokang
xiaokang approved these changes May 16, 2024
View reviewed changes

@xiaokangxiaokang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring
dataroaring merged commit a17214d into apache:masterMay 16, 2024
@airborne12
airborne12 deleted the fix-race branch May 16, 2024 06:30
airborne12 added a commit to airborne12/apache-doris that referenced this pull request May 16, 2024
@airborne12
[Fix](inverted index) fix race condition for column reader load inver…
07cdd75
…ted index reader (apache#34922)
@airborne12airborne12 mentioned this pull request May 16, 2024
Merged
dataroaring pushed a commit that referenced this pull request May 16, 2024
@airborne12@dataroaring
[Fix](inverted index) fix race condition for column reader load inver…
b76b3e9
…ted index reader (#34922)
xiaokang pushed a commit that referenced this pull request May 17, 2024
@airborne12
[fix](inverted index) fix race condition for column reader load inver…
5871e91
…ted index reader #34922 (#34969)
airborne12 added a commit to airborne12/apache-doris that referenced this pull request May 20, 2024
@airborne12
[Fix](inverted index) fix race condition for column reader load inver…
60d4d9d
…ted index reader (apache#34922)
@airborne12airborne12 mentioned this pull request May 20, 2024
Merged
yiguolei pushed a commit that referenced this pull request May 20, 2024
@airborne12
[Fix](inverted index) fix race condition for column reader load inver…
7aff10b
…ted index reader (#34922) (#35040)
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@airborne12weixingyu12
[fix](inverted index) fix race condition for column reader load inver…
f8996b5
…ted index reader apache#34922 (apache#34969)
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@airborne12@HappenLee
[Fix](inverted index) fix race condition for column reader load inver…
3a413e3
…ted index reader (apache#34922)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xiaokangxiaokangxiaokang approved these changes
@eldenmooneldenmooneldenmoon approved these changes
@dataroaringdataroaringdataroaring approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.0.11-mergeddev/2.1.4-mergeddev/3.0.0-mergedp0_breviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@airborne12@doris-robot@dataroaring@xiaokang@eldenmoon@yiguolei