Skip to content

[opt](scanner profile) Rename some filed name to keep consistent with audit log. - #41993

Merged
yiguolei merged 1 commit into
apache:masterfrom
RoanHeNaN:fix-scan-rows
Oct 19, 2024
Merged

[opt](scanner profile) Rename some filed name to keep consistent with audit log.#41993
yiguolei merged 1 commit into
apache:masterfrom
RoanHeNaN:fix-scan-rows

Conversation

@RoanHeNaN

@RoanHeNaNRoanHeNaN commented Oct 16, 2024

Copy link
Copy Markdown
Contributor

RawRowsRead is moved from segment profile to scan operator profile.
RawRowsRead is renamed to ScanRows so that it has same name with audit log.
ScanBytes is added. It has same value with CompressedBytesRead in segment iterator profile

image

I am not sure whether we should modify the calculation of CompressedBytesRead.
Currently, CompressedBytesRead does not include size of page that is cached by doris page cache.

@RoanHeNaN
NEED
2745f12
NEED
@doris-robot

doris-robot commented Oct 16, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@RoanHeNaN

RoanHeNaN commented Oct 16, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Oct 16, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot

doris-robot commented Oct 16, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 37.47% (9711/25919)
Line Coverage: 28.73% (80613/280616)
Region Coverage: 28.18% (41726/148088)
Branch Coverage: 24.76% (21217/85698)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2745f12d09b64dbb098d41e651d9cbdb032350f9_2745f12d09b64dbb098d41e651d9cbdb032350f9/report/index.html

@doris-robot

doris-robot commented Oct 16, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 41629 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2745f12d09b64dbb098d41e651d9cbdb032350f9, data reload: false
------ Round 1 ----------------------------------
q1	17610	7454	7341	7341
q2	2046	283	283	283
q3	12264	1083	1206	1083
q4	10575	815	949	815
q5	7780	3117	3101	3101
q6	236	153	153	153
q7	1033	619	629	619
q8	9373	1945	2021	1945
q9	6655	6492	6454	6454
q10	7078	2455	2543	2455
q11	445	242	243	242
q12	440	224	228	224
q13	17773	3013	2981	2981
q14	238	208	213	208
q15	569	514	509	509
q16	654	596	577	577
q17	982	528	542	528
q18	7218	6739	6798	6739
q19	1352	1049	927	927
q20	499	189	190	189
q21	4090	3243	3301	3243
q22	1111	1013	1016	1013
Total cold run time: 110021 ms
Total hot run time: 41629 ms
----- Round 2, with runtime_filter_mode=off -----
q1	7357	7427	7298	7298
q2	341	239	239	239
q3	3051	2993	3018	2993
q4	2074	1855	1799	1799
q5	5778	5769	5830	5769
q6	243	146	155	146
q7	2276	1842	1813	1813
q8	3418	3490	3523	3490
q9	8952	8938	8902	8902
q10	3623	3601	3526	3526
q11	610	496	493	493
q12	859	634	637	634
q13	11055	3215	3201	3201
q14	290	295	277	277
q15	590	528	518	518
q16	681	644	674	644
q17	1864	1611	1604	1604
q18	8274	7934	7728	7728
q19	1749	1642	1468	1468
q20	2122	1891	1894	1891
q21	5722	5389	5453	5389
q22	1114	1094	1070	1070
Total cold run time: 72043 ms
Total hot run time: 60892 ms

@doris-robot

doris-robot commented Oct 16, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 192222 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2745f12d09b64dbb098d41e651d9cbdb032350f9, data reload: false
query1	914	387	404	387
query2	6244	2135	2112	2112
query3	8678	195	201	195
query4	34408	23705	23513	23513
query5	3484	488	477	477
query6	279	166	164	164
query7	4187	297	289	289
query8	291	221	232	221
query9	9339	2736	2723	2723
query10	483	278	266	266
query11	17859	15327	15120	15120
query12	146	102	100	100
query13	1592	446	403	403
query14	9083	7379	7470	7379
query15	241	169	179	169
query16	7911	461	452	452
query17	1676	613	606	606
query18	2007	324	315	315
query19	372	170	166	166
query20	132	114	114	114
query21	225	106	108	106
query22	4879	4579	4484	4484
query23	35027	34094	34179	34094
query24	10906	2773	2777	2773
query25	638	408	409	408
query26	1210	161	162	161
query27	2273	295	289	289
query28	7413	2430	2418	2418
query29	825	432	424	424
query30	279	152	150	150
query31	1069	824	779	779
query32	99	55	56	55
query33	770	296	300	296
query34	920	512	517	512
query35	917	759	739	739
query36	1103	960	955	955
query37	170	92	82	82
query38	4092	3874	3961	3874
query39	1491	1429	1414	1414
query40	210	100	99	99
query41	50	46	48	46
query42	120	103	100	100
query43	547	494	494	494
query44	1250	832	821	821
query45	200	165	168	165
query46	1149	729	737	729
query47	1965	1869	1872	1869
query48	422	310	329	310
query49	986	449	443	443
query50	822	382	388	382
query51	7204	6969	7015	6969
query52	105	89	86	86
query53	260	184	183	183
query54	1120	439	432	432
query55	82	80	81	80
query56	297	274	288	274
query57	1294	1143	1139	1139
query58	246	225	225	225
query59	3294	3015	2933	2933
query60	292	256	265	256
query61	105	97	125	97
query62	898	667	688	667
query63	226	195	181	181
query64	4063	649	639	639
query65	3276	3153	3191	3153
query66	1293	308	296	296
query67	15785	15756	15573	15573
query68	4342	571	586	571
query69	504	284	285	284
query70	1177	1132	1064	1064
query71	387	273	280	273
query72	7304	4008	4007	4007
query73	785	361	364	361
query74	9771	8991	8985	8985
query75	3453	2662	2645	2645
query76	2741	841	880	841
query77	598	305	293	293
query78	11181	10457	9804	9804
query79	2554	614	619	614
query80	1516	445	450	445
query81	558	247	239	239
query82	574	142	138	138
query83	300	132	139	132
query84	279	69	72	69
query85	798	298	281	281
query86	484	307	287	287
query87	4499	4443	4337	4337
query88	3620	2217	2180	2180
query89	408	297	283	283
query90	2064	185	184	184
query91	148	100	104	100
query92	69	50	48	48
query93	1715	539	539	539
query94	1096	295	299	295
query95	356	246	256	246
query96	625	279	281	279
query97	3247	3082	3139	3082
query98	231	203	197	197
query99	1558	1294	1306	1294
Total cold run time: 300135 ms
Total hot run time: 192222 ms

@doris-robot

doris-robot commented Oct 16, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 33.9 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2745f12d09b64dbb098d41e651d9cbdb032350f9, data reload: false
query1	0.04	0.04	0.03
query2	0.06	0.03	0.03
query3	0.23	0.06	0.06
query4	1.65	0.10	0.10
query5	0.51	0.48	0.53
query6	1.14	0.73	0.73
query7	0.02	0.02	0.01
query8	0.03	0.03	0.02
query9	0.56	0.50	0.50
query10	0.55	0.56	0.53
query11	0.14	0.11	0.11
query12	0.15	0.11	0.10
query13	0.60	0.58	0.59
query14	2.81	2.77	2.86
query15	0.90	0.83	0.84
query16	0.39	0.37	0.38
query17	1.02	1.02	0.99
query18	0.20	0.19	0.20
query19	1.86	1.73	2.01
query20	0.01	0.01	0.01
query21	15.35	0.62	0.59
query22	2.91	2.49	2.52
query23	16.82	0.94	0.76
query24	3.19	1.99	1.68
query25	0.30	0.18	0.10
query26	0.56	0.14	0.13
query27	0.04	0.04	0.04
query28	9.23	1.09	1.06
query29	12.57	3.22	3.19
query30	0.24	0.06	0.07
query31	2.86	0.38	0.38
query32	3.27	0.46	0.47
query33	3.04	3.01	3.02
query34	16.97	4.57	4.46
query35	4.48	4.59	4.55
query36	0.67	0.47	0.50
query37	0.08	0.06	0.06
query38	0.04	0.03	0.04
query39	0.03	0.03	0.02
query40	0.15	0.12	0.12
query41	0.07	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.82 s
Total hot run time: 33.9 s

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Oct 19, 2024
@github-actions

github-actionsBot commented Oct 19, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Oct 19, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@yiguolei
yiguolei merged commit 5c1113b into apache:masterOct 19, 2024
@RoanHeNaN
RoanHeNaN deleted the fix-scan-rows branch October 19, 2024 10:32
RoanHeNaN added a commit to RoanHeNaN/doris that referenced this pull request Oct 21, 2024
@RoanHeNaN
[opt](scanner profile) Rename some filed name to keep consistent with…
4e48e7c
… audit log. (apache#41993)
@RoanHeNaNRoanHeNaN mentioned this pull request Oct 21, 2024
Merged
RoanHeNaN added a commit to RoanHeNaN/doris that referenced this pull request Oct 21, 2024
@RoanHeNaN
[opt](scanner profile) Rename some filed name to keep consistent with…
17e7215
… audit log. (apache#41993)
@RoanHeNaNRoanHeNaN mentioned this pull request Oct 21, 2024
Merged
yiguolei pushed a commit that referenced this pull request Oct 24, 2024
@RoanHeNaN
[opt](scanner profile) Rename some filed name to keep consistent with…
972c37d
… audit log. #41993 (#42137)
cherry pick from #41993
yiguolei pushed a commit that referenced this pull request Oct 25, 2024
@RoanHeNaN
[opt](scanner profile) Rename some filed name to keep consistent with…
8ae348a
… audit log. #41993 (#42138)
cherry pick from #41993
@yiguoleiyiguolei mentioned this pull request Nov 6, 2024
Closed
@yiguoleiyiguolei mentioned this pull request Jan 19, 2025
Closed
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@RoanHeNaN@HappenLee
[opt](scanner profile) Rename some filed name to keep consistent with…
d30f1f7
… audit log. (apache#41993)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningmanmorningmanmorningman approved these changes
@yiguoleiyiguoleiyiguolei approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.1.7-mergeddev/3.0.3-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@RoanHeNaN@doris-robot@morningman@yiguolei