Skip to content

Revert "[fix](scanner) Fix incorrect _max_thread_num in scanner context" - #40804

Merged
yiguolei merged 1 commit into
apache:masterfrom
RoanHeNaN:revert-40569-refactor-scanner
Sep 16, 2024
Merged

Revert "[fix](scanner) Fix incorrect _max_thread_num in scanner context"#40804
yiguolei merged 1 commit into
apache:masterfrom
RoanHeNaN:revert-40569-refactor-scanner

Conversation

@RoanHeNaN

@RoanHeNaNRoanHeNaN commented Sep 13, 2024

Copy link
Copy Markdown
Contributor

Reverts #40569

We need more test to avoid performance issue

@doris-robot

doris-robot commented Sep 13, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@RoanHeNaN

RoanHeNaN commented Sep 13, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Sep 13, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 36.89% (9463/25654)
Line Coverage: 28.25% (77815/275424)
Region Coverage: 27.66% (40182/145296)
Branch Coverage: 24.25% (20406/84140)
Coverage Report: http://coverage.selectdb-in.cc/coverage/397ee86830022674831fa2154882a531fa338a08_397ee86830022674831fa2154882a531fa338a08/report/index.html

@doris-robot

doris-robot commented Sep 13, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 41546 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 397ee86830022674831fa2154882a531fa338a08, data reload: false
------ Round 1 ----------------------------------
q1	17762	7593	7494	7494
q2	2010	194	186	186
q3	11774	974	1180	974
q4	10520	694	787	694
q5	7754	2875	2804	2804
q6	238	151	150	150
q7	976	622	608	608
q8	9541	2083	2048	2048
q9	7817	6592	6637	6592
q10	7040	2174	2247	2174
q11	459	242	258	242
q12	413	218	217	217
q13	17755	3089	3079	3079
q14	283	239	248	239
q15	591	550	534	534
q16	542	442	427	427
q17	983	679	753	679
q18	7476	6954	6951	6951
q19	1376	1054	1038	1038
q20	695	352	343	343
q21	3948	3050	3172	3050
q22	1109	1024	1023	1023
Total cold run time: 111062 ms
Total hot run time: 41546 ms
----- Round 2, with runtime_filter_mode=off -----
q1	7574	7584	7584	7584
q2	380	279	284	279
q3	3106	2876	2869	2869
q4	2062	1765	1703	1703
q5	5723	5628	5805	5628
q6	241	147	144	144
q7	2194	1869	1868	1868
q8	3326	3479	3494	3479
q9	8958	8834	8827	8827
q10	3630	3399	3422	3399
q11	625	527	529	527
q12	842	660	658	658
q13	13756	3315	3329	3315
q14	321	284	288	284
q15	596	539	532	532
q16	546	482	490	482
q17	1854	1597	1551	1551
q18	8294	7923	8027	7923
q19	1741	1637	1668	1637
q20	2146	1925	1933	1925
q21	5562	5339	5471	5339
q22	1153	1108	1057	1057
Total cold run time: 74630 ms
Total hot run time: 61010 ms

HappenLee
HappenLee approved these changes Sep 13, 2024
View reviewed changes

@HappenLeeHappenLee left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 13, 2024
@github-actions

github-actionsBot commented Sep 13, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Sep 13, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@yiguolei
yiguoleiforce-pushed the revert-40569-refactor-scanner branch from 397ee86 to a551f8dCompareSeptember 15, 2024 15:11
@yiguolei

yiguolei commented Sep 15, 2024

Copy link
Copy Markdown
Contributor

run buildall

@doris-robot

doris-robot commented Sep 15, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 41243 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a551f8ddc3ea905dee61d2be52030a50edb1850e, data reload: false
------ Round 1 ----------------------------------
q1	17564	7382	7259	7259
q2	2043	170	151	151
q3	10650	1094	1174	1094
q4	10556	762	716	716
q5	7760	3052	3063	3052
q6	225	148	142	142
q7	1003	608	598	598
q8	9417	2032	2036	2032
q9	6826	6420	6390	6390
q10	7065	2286	2300	2286
q11	446	249	246	246
q12	398	209	218	209
q13	17778	2977	2994	2977
q14	238	205	233	205
q15	560	528	525	525
q16	498	419	421	419
q17	989	805	830	805
q18	7174	6723	6650	6650
q19	1394	1126	1079	1079
q20	557	279	270	270
q21	3946	3147	3122	3122
q22	1115	1029	1016	1016
Total cold run time: 108202 ms
Total hot run time: 41243 ms
----- Round 2, with runtime_filter_mode=off -----
q1	7217	7181	7166	7166
q2	324	246	224	224
q3	2945	3006	2983	2983
q4	2024	1816	1834	1816
q5	5569	5620	5599	5599
q6	223	140	149	140
q7	2180	1827	1761	1761
q8	3296	3508	3478	3478
q9	8761	8860	8778	8778
q10	3531	3450	3462	3450
q11	593	488	476	476
q12	807	615	640	615
q13	10650	3184	3109	3109
q14	297	273	280	273
q15	584	533	512	512
q16	520	459	462	459
q17	1787	1608	1580	1580
q18	8313	7701	7744	7701
q19	1709	1625	1547	1547
q20	2095	1884	1900	1884
q21	5591	5218	5410	5218
q22	1174	1076	1069	1069
Total cold run time: 70190 ms
Total hot run time: 59838 ms

@doris-robot

doris-robot commented Sep 15, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 199291 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a551f8ddc3ea905dee61d2be52030a50edb1850e, data reload: false
query1	1319	872	909	872
query2	6376	2039	2014	2014
query3	10780	3895	3849	3849
query4	62763	29148	23557	23557
query5	5085	474	463	463
query6	399	172	161	161
query7	5419	310	289	289
query8	312	223	220	220
query9	8590	2625	2602	2602
query10	435	294	279	279
query11	17223	15080	15673	15080
query12	149	115	103	103
query13	1467	414	404	404
query14	10773	7508	7538	7508
query15	208	187	176	176
query16	6807	500	511	500
query17	1157	626	579	579
query18	1686	326	306	306
query19	208	154	151	151
query20	127	110	109	109
query21	204	109	105	105
query22	5065	4576	4755	4576
query23	34700	34287	34168	34168
query24	6026	2887	2881	2881
query25	514	395	408	395
query26	633	155	161	155
query27	1693	283	278	278
query28	4031	2148	2110	2110
query29	662	422	427	422
query30	222	146	155	146
query31	947	769	771	769
query32	71	53	51	51
query33	393	295	298	295
query34	901	498	491	491
query35	827	737	739	737
query36	1044	928	937	928
query37	134	75	79	75
query38	4003	3940	3939	3939
query39	1460	1384	1419	1384
query40	208	97	94	94
query41	48	47	48	47
query42	114	95	99	95
query43	514	492	489	489
query44	1132	815	779	779
query45	195	166	168	166
query46	1122	778	752	752
query47	1924	1833	1860	1833
query48	467	365	367	365
query49	714	403	407	403
query50	845	500	404	404
query51	7071	6832	6927	6832
query52	95	84	87	84
query53	256	177	179	177
query54	545	439	454	439
query55	76	75	72	72
query56	274	258	260	258
query57	1210	1119	1082	1082
query58	211	219	226	219
query59	3304	3149	3001	3001
query60	275	262	258	258
query61	103	113	97	97
query62	761	664	649	649
query63	216	183	182	182
query64	1336	622	631	622
query65	3261	3170	3164	3164
query66	678	297	301	297
query67	16166	15476	15775	15476
query68	1468	868	849	849
query69	447	338	337	337
query70	1198	1133	1179	1133
query71	326	336	329	329
query72	5838	3390	3397	3390
query73	588	581	582	581
query74	9239	9026	9071	9026
query75	2908	2923	2884	2884
query76	1073	873	863	863
query77	416	358	364	358
query78	9482	9283	9502	9283
query79	921	890	863	863
query80	585	561	555	555
query81	445	282	235	235
query82	195	192	194	192
query83	162	159	153	153
query84	263	110	92	92
query85	660	390	343	343
query86	312	324	303	303
query87	4496	4377	4429	4377
query88	4550	4014	4008	4008
query89	368	359	361	359
query90	1382	307	299	299
query91	163	177	159	159
query92	74	72	71	71
query93	921	895	895	895
query94	502	345	382	345
query95	415	419	454	419
query96	478	481	474	474
query97	3130	3125	3141	3125
query98	234	223	232	223
query99	1418	1321	1305	1305
Total cold run time: 302591 ms
Total hot run time: 199291 ms

@doris-robot

doris-robot commented Sep 15, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.89 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a551f8ddc3ea905dee61d2be52030a50edb1850e, data reload: false
query1	0.05	0.04	0.04
query2	0.06	0.03	0.03
query3	0.23	0.07	0.06
query4	1.64	0.10	0.10
query5	0.51	0.50	0.50
query6	1.14	0.73	0.73
query7	0.02	0.01	0.02
query8	0.04	0.03	0.04
query9	0.56	0.50	0.48
query10	0.56	0.57	0.55
query11	0.16	0.11	0.10
query12	0.14	0.11	0.11
query13	0.60	0.59	0.58
query14	1.34	1.43	1.41
query15	0.83	0.82	0.81
query16	0.37	0.40	0.38
query17	1.06	1.07	1.01
query18	0.20	0.19	0.19
query19	1.90	1.81	1.76
query20	0.01	0.01	0.02
query21	15.39	0.59	0.58
query22	2.54	2.36	2.67
query23	17.04	0.80	0.87
query24	2.74	1.54	1.27
query25	0.19	0.14	0.14
query26	0.52	0.13	0.13
query27	0.05	0.04	0.04
query28	10.57	1.08	1.06
query29	12.53	3.27	3.25
query30	0.25	0.05	0.06
query31	2.89	0.38	0.37
query32	3.27	0.46	0.46
query33	2.99	2.94	3.05
query34	16.53	4.45	4.38
query35	4.46	4.40	4.47
query36	0.66	0.50	0.48
query37	0.08	0.06	0.05
query38	0.04	0.03	0.03
query39	0.04	0.02	0.02
query40	0.16	0.13	0.13
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 104.49 s
Total hot run time: 31.89 s

@doris-robot

doris-robot commented Sep 15, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 37.31% (9575/25662)
Line Coverage: 28.68% (79069/275726)
Region Coverage: 28.17% (40950/145390)
Branch Coverage: 24.79% (20872/84188)
Coverage Report: http://coverage.selectdb-in.cc/coverage/a551f8ddc3ea905dee61d2be52030a50edb1850e_a551f8ddc3ea905dee61d2be52030a50edb1850e/report/index.html

@yiguolei
yiguolei merged commit b92b63e into apache:masterSep 16, 2024
dataroaring pushed a commit that referenced this pull request Oct 9, 2024
@RoanHeNaN@dataroaring
Revert "[fix](scanner) Fix incorrect _max_thread_num in scanner conte…
18588c5
…xt" (#40804)
Reverts #40569
We need more test to avoid performance issue
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@RoanHeNaN@HappenLee
Revert "[fix](scanner) Fix incorrect _max_thread_num in scanner conte…
38b7d17
…xt" (apache#40804)
Reverts apache#40569
We need more test to avoid performance issue
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguoleiyiguoleiyiguolei approved these changes
@HappenLeeHappenLeeHappenLee approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/3.0.3-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@RoanHeNaN@doris-robot@yiguolei@HappenLee@dataroaring