Skip to content

[feature](metacache) add system table catalog_meta_cache_statistics - #40155

Merged
morningman merged 9 commits into
apache:masterfrom
morningman:meta_cache_table
Sep 3, 2024
Merged

[feature](metacache) add system table catalog_meta_cache_statistics#40155
morningman merged 9 commits into
apache:masterfrom
morningman:meta_cache_table

Conversation

@morningman

@morningmanmorningman commented Aug 29, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Add new system table catalog_meta_cache_statistics

mysql> select * from information_schema.catalog_meta_cache_statistics;
+--------------+-----------------------------+----------------------+---------------------+
| CATALOG_NAME | CACHE_NAME | METRIC_NAME | METRIC_VALUE |
+--------------+-----------------------------+----------------------+---------------------+
| hms | hive_file_cache | eviction_count | 0 |
| hms | hive_file_cache | hit_ratio | 0.6 |
| hms | hive_file_cache | average_load_penalty | 7.60805755E7 |
| hms | hive_file_cache | estimated_size | 2 |
| hms | hive_file_cache | hit_count | 3 |
| hms | hive_file_cache | read_count | 5 |
| hms | hive_partition_cache | eviction_count | 0 |
| hms | hive_partition_cache | hit_ratio | 1.0 |
| hms | hive_partition_cache | average_load_penalty | 0.0 |
| hms | hive_partition_cache | estimated_size | 0 |
| hms | hive_partition_cache | hit_count | 0 |
| hms | hive_partition_cache | read_count | 0 |
| hms | hive_partition_values_cache | eviction_count | 0 |
| hms | hive_partition_values_cache | hit_ratio | 1.0 |
| hms | hive_partition_values_cache | average_load_penalty | 0.0 |
| hms | hive_partition_values_cache | estimated_size | 0 |
| hms | hive_partition_values_cache | hit_count | 0 |
| hms | hive_partition_values_cache | read_count | 0 |
| hms | hudi_partition_cache | eviction_count | 0 |
| hms | hudi_partition_cache | hit_ratio | 1.0 |
| hms | hudi_partition_cache | average_load_penalty | 0.0 |
| hms | hudi_partition_cache | estimated_size | 0 |
| hms | hudi_partition_cache | hit_count | 0 |
| hms | hudi_partition_cache | read_count | 0 |
| iceberg | iceberg_table_cache | eviction_count | 0 |
| iceberg | iceberg_table_cache | hit_ratio | 0.07407407407407407 |
| iceberg | iceberg_table_cache | average_load_penalty | 1.7263263424E8 |
| iceberg | iceberg_table_cache | estimated_size | 25 |
| iceberg | iceberg_table_cache | hit_count | 2 |
| iceberg | iceberg_table_cache | read_count | 27 |
| iceberg | iceberg_snapshot_cache | eviction_count | 0 |
| iceberg | iceberg_snapshot_cache | hit_ratio | 1.0 |
| iceberg | iceberg_snapshot_cache | average_load_penalty | 0.0 |
| iceberg | iceberg_snapshot_cache | estimated_size | 0 |
| iceberg | iceberg_snapshot_cache | hit_count | 0 |
| iceberg | iceberg_snapshot_cache | read_count | 0 |
+--------------+-----------------------------+----------------------+---------------------+

@doris-robot

doris-robot commented Aug 29, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions

github-actionsBot commented Aug 29, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

3 similar comments
@github-actions

github-actionsBot commented Aug 30, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions

github-actionsBot commented Aug 30, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions

github-actionsBot commented Aug 30, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

github-actions[bot]
github-actionsBot reviewed Aug 30, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/exec/schema_scanner/schema_catalog_meta_cache_stats_scanner.cpp
Comment threadbe/src/exec/schema_scanner/schema_catalog_meta_cache_stats_scanner.cpp
@morningman

morningman commented Aug 30, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Aug 30, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 38064 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 282d7c3055e9267956b50958650a7172c8be72b8, data reload: false
------ Round 1 ----------------------------------
q1	17681	4661	4359	4359
q2	2044	187	175	175
q3	11774	1002	1066	1002
q4	10519	700	719	700
q5	7779	2827	2833	2827
q6	231	140	141	140
q7	978	614	608	608
q8	9332	2102	2101	2101
q9	7266	6559	6559	6559
q10	7000	2164	2245	2164
q11	444	246	241	241
q12	411	220	234	220
q13	17770	3049	3007	3007
q14	288	241	244	241
q15	544	477	494	477
q16	585	510	511	510
q17	997	728	751	728
q18	7524	6899	6857	6857
q19	1386	1019	1078	1019
q20	719	326	340	326
q21	4120	2812	2946	2812
q22	1131	1048	991	991
Total cold run time: 110523 ms
Total hot run time: 38064 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4352	4380	4332	4332
q2	381	276	270	270
q3	2914	2684	2631	2631
q4	1943	1675	1688	1675
q5	5629	5740	5741	5740
q6	232	144	144	144
q7	2248	1862	1857	1857
q8	3381	3440	3512	3440
q9	8854	8831	8814	8814
q10	3646	3419	3381	3381
q11	624	505	513	505
q12	841	669	665	665
q13	14719	3081	3268	3081
q14	321	285	291	285
q15	573	498	511	498
q16	638	578	568	568
q17	1869	1563	1540	1540
q18	8072	7782	8011	7782
q19	1729	1607	1674	1607
q20	2162	1907	1897	1897
q21	5950	5619	5620	5619
q22	1132	1065	1005	1005
Total cold run time: 72210 ms
Total hot run time: 57336 ms

@doris-robot

doris-robot commented Aug 30, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 193673 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 282d7c3055e9267956b50958650a7172c8be72b8, data reload: false
query1	1263	902	875	875
query2	6311	2018	2063	2018
query3	10684	4366	4133	4133
query4	59802	23788	23176	23176
query5	5471	497	496	496
query6	410	172	181	172
query7	5781	294	298	294
query8	292	215	216	215
query9	8719	2499	2490	2490
query10	490	281	270	270
query11	18352	14995	15311	14995
query12	164	104	104	104
query13	1556	410	379	379
query14	10428	7485	7494	7485
query15	267	178	181	178
query16	7978	437	499	437
query17	1380	602	564	564
query18	1978	295	297	295
query19	287	151	146	146
query20	122	110	112	110
query21	211	105	109	105
query22	4699	4569	4932	4569
query23	34320	33596	33230	33230
query24	5985	2837	2841	2837
query25	523	382	380	380
query26	689	158	163	158
query27	1771	286	280	280
query28	3869	2131	2108	2108
query29	678	406	440	406
query30	241	154	157	154
query31	935	777	782	777
query32	87	54	62	54
query33	450	285	284	284
query34	868	479	498	479
query35	853	721	696	696
query36	1072	912	950	912
query37	155	95	90	90
query38	4046	3852	3896	3852
query39	1451	1394	1417	1394
query40	196	118	117	117
query41	47	49	45	45
query42	121	98	96	96
query43	516	502	496	496
query44	1123	756	757	756
query45	195	163	167	163
query46	1094	788	741	741
query47	1923	1819	1803	1803
query48	369	301	294	294
query49	750	434	435	434
query50	810	422	430	422
query51	7158	7041	6932	6932
query52	102	93	89	89
query53	252	182	183	182
query54	584	459	467	459
query55	78	77	81	77
query56	287	278	282	278
query57	1209	1061	1100	1061
query58	231	239	238	238
query59	3199	2932	2886	2886
query60	311	279	277	277
query61	128	118	123	118
query62	781	652	659	652
query63	221	182	186	182
query64	2846	685	706	685
query65	3240	3155	3128	3128
query66	618	361	333	333
query67	15405	15195	15466	15195
query68	2906	588	598	588
query69	404	291	278	278
query70	1147	1106	1121	1106
query71	333	280	279	279
query72	6154	4136	3986	3986
query73	755	334	335	334
query74	9199	8725	8853	8725
query75	3368	2686	2749	2686
query76	1497	1078	992	992
query77	554	328	320	320
query78	10796	9286	9279	9279
query79	1035	552	542	542
query80	725	513	499	499
query81	545	236	235	235
query82	244	148	151	148
query83	170	148	149	148
query84	278	78	77	77
query85	681	342	278	278
query86	300	270	287	270
query87	4357	4315	4296	4296
query88	3260	2320	2326	2320
query89	377	296	288	288
query90	1995	200	202	200
query91	123	98	98	98
query92	60	50	62	50
query93	1051	536	539	536
query94	678	300	301	300
query95	357	267	265	265
query96	604	269	269	269
query97	3175	3060	3069	3060
query98	220	204	199	199
query99	1486	1286	1299	1286
Total cold run time: 307266 ms
Total hot run time: 193673 ms

@doris-robot

doris-robot commented Aug 30, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 32.41 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 282d7c3055e9267956b50958650a7172c8be72b8, data reload: false
query1	0.04	0.05	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.67	0.09	0.09
query5	0.48	0.50	0.50
query6	1.12	0.74	0.74
query7	0.02	0.02	0.01
query8	0.05	0.05	0.05
query9	0.55	0.48	0.49
query10	0.56	0.54	0.55
query11	0.16	0.12	0.12
query12	0.15	0.13	0.13
query13	0.62	0.60	0.59
query14	2.08	2.11	2.06
query15	0.85	0.83	0.84
query16	0.38	0.37	0.39
query17	1.02	1.02	1.05
query18	0.21	0.20	0.20
query19	1.82	1.70	1.73
query20	0.01	0.01	0.02
query21	15.40	0.68	0.65
query22	4.46	6.72	1.96
query23	18.27	1.35	1.34
query24	2.07	0.22	0.22
query25	0.13	0.08	0.08
query26	0.27	0.19	0.18
query27	0.08	0.08	0.07
query28	13.32	1.02	1.01
query29	12.60	3.50	3.45
query30	0.25	0.06	0.05
query31	2.87	0.41	0.40
query32	3.23	0.48	0.48
query33	2.96	2.98	2.99
query34	16.98	4.45	4.42
query35	4.52	4.52	4.46
query36	0.68	0.49	0.50
query37	0.20	0.16	0.16
query38	0.16	0.14	0.15
query39	0.04	0.04	0.04
query40	0.16	0.13	0.13
query41	0.10	0.05	0.05
query42	0.06	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 110.95 s
Total hot run time: 32.41 s

@morningman

morningman commented Aug 31, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@morningman
morningman marked this pull request as ready for review August 31, 2024 02:07
@morningmanmorningman mentioned this pull request Aug 31, 2024
Merged
@doris-robot

doris-robot commented Aug 31, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 38142 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d651001442c1cfce8cfa287ca30338c2757973f7, data reload: false
------ Round 1 ----------------------------------
q1	17614	4531	4346	4346
q2	2026	183	177	177
q3	11699	946	1172	946
q4	10509	760	708	708
q5	7764	2843	2845	2843
q6	228	139	138	138
q7	949	605	605	605
q8	9320	2046	2052	2046
q9	7124	6542	6517	6517
q10	6991	2230	2227	2227
q11	458	238	241	238
q12	393	222	229	222
q13	17974	2984	3015	2984
q14	284	256	235	235
q15	508	492	486	486
q16	570	502	506	502
q17	974	666	687	666
q18	7367	6896	6919	6896
q19	1394	1053	1005	1005
q20	697	334	336	334
q21	3891	3027	2992	2992
q22	1137	1035	1029	1029
Total cold run time: 109871 ms
Total hot run time: 38142 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4408	4421	4292	4292
q2	369	278	275	275
q3	2923	2686	2717	2686
q4	1940	1634	1683	1634
q5	5613	5665	5737	5665
q6	241	142	138	138
q7	2221	1832	1834	1832
q8	3298	3396	3446	3396
q9	8791	8754	8773	8754
q10	3547	3352	3355	3352
q11	614	500	523	500
q12	845	640	675	640
q13	13713	3202	3207	3202
q14	338	296	282	282
q15	535	500	517	500
q16	614	579	567	567
q17	1825	1557	1505	1505
q18	8160	7823	7925	7823
q19	1723	1508	1621	1508
q20	2183	1929	1913	1913
q21	5717	5535	5551	5535
q22	1138	1050	1023	1023
Total cold run time: 70756 ms
Total hot run time: 57022 ms

@doris-robot

doris-robot commented Aug 31, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 193560 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d651001442c1cfce8cfa287ca30338c2757973f7, data reload: false
query1	1282	896	858	858
query2	6339	2018	2045	2018
query3	10728	4067	4086	4067
query4	60324	23820	23327	23327
query5	5464	497	504	497
query6	400	162	173	162
query7	5776	305	305	305
query8	284	204	201	201
query9	8849	2492	2470	2470
query10	472	272	257	257
query11	16927	15095	15149	15095
query12	165	113	114	113
query13	1562	393	389	389
query14	11107	7852	7410	7410
query15	227	178	180	178
query16	7452	430	524	430
query17	1164	583	552	552
query18	1834	295	297	295
query19	290	150	147	147
query20	117	115	113	113
query21	208	104	101	101
query22	4547	4660	4333	4333
query23	34282	33638	33685	33638
query24	5948	2923	2934	2923
query25	529	371	381	371
query26	678	161	154	154
query27	1767	297	283	283
query28	3949	2118	2102	2102
query29	654	403	409	403
query30	231	155	158	155
query31	931	723	794	723
query32	77	51	56	51
query33	390	273	289	273
query34	867	493	510	493
query35	833	711	737	711
query36	1031	953	933	933
query37	146	91	87	87
query38	3932	3812	3970	3812
query39	1453	1404	1373	1373
query40	193	115	115	115
query41	48	45	49	45
query42	108	97	100	97
query43	507	489	477	477
query44	1099	754	748	748
query45	206	174	171	171
query46	1102	755	748	748
query47	1932	1818	1829	1818
query48	379	306	301	301
query49	785	447	482	447
query50	817	435	426	426
query51	7176	7001	6993	6993
query52	95	90	96	90
query53	250	179	180	179
query54	566	464	446	446
query55	79	76	74	74
query56	268	257	246	246
query57	1172	1047	1098	1047
query58	226	226	233	226
query59	2984	2826	2920	2826
query60	301	265	271	265
query61	103	95	101	95
query62	745	657	664	657
query63	212	188	187	187
query64	2830	678	655	655
query65	3227	3141	3139	3139
query66	674	341	348	341
query67	15508	15160	15249	15160
query68	2965	571	571	571
query69	410	284	281	281
query70	1186	1144	1138	1138
query71	333	273	274	273
query72	5338	4012	3959	3959
query73	750	332	337	332
query74	9098	8806	8804	8804
query75	3369	2750	2705	2705
query76	1323	1002	994	994
query77	520	337	335	335
query78	9900	9071	9012	9012
query79	1036	537	538	537
query80	688	516	511	511
query81	468	235	233	233
query82	237	149	145	145
query83	171	151	151	151
query84	254	80	86	80
query85	685	292	282	282
query86	306	269	290	269
query87	4422	4258	4302	4258
query88	2954	2327	2312	2312
query89	382	283	284	283
query90	2068	199	198	198
query91	125	100	114	100
query92	64	54	55	54
query93	1041	540	541	540
query94	744	320	310	310
query95	344	271	271	271
query96	602	273	269	269
query97	3239	3050	3068	3050
query98	219	208	216	208
query99	1507	1273	1301	1273
Total cold run time: 303556 ms
Total hot run time: 193560 ms

@doris-robot

doris-robot commented Aug 31, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 32.41 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d651001442c1cfce8cfa287ca30338c2757973f7, data reload: false
query1	0.04	0.05	0.04
query2	0.09	0.03	0.04
query3	0.23	0.05	0.05
query4	1.68	0.08	0.08
query5	0.50	0.49	0.48
query6	1.14	0.73	0.74
query7	0.02	0.02	0.01
query8	0.06	0.04	0.05
query9	0.56	0.49	0.48
query10	0.55	0.53	0.53
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.63	0.60	0.59
query14	2.09	2.05	2.11
query15	0.85	0.84	0.81
query16	0.37	0.38	0.37
query17	1.06	1.01	0.97
query18	0.21	0.21	0.21
query19	1.87	1.76	1.80
query20	0.02	0.01	0.01
query21	15.40	0.67	0.67
query22	4.50	6.30	2.23
query23	18.22	1.40	1.28
query24	2.13	0.24	0.22
query25	0.15	0.08	0.08
query26	0.28	0.17	0.18
query27	0.08	0.08	0.07
query28	13.23	1.01	1.00
query29	12.62	3.46	3.44
query30	0.24	0.05	0.04
query31	2.87	0.40	0.40
query32	3.23	0.50	0.47
query33	2.98	2.97	3.00
query34	17.16	4.40	4.43
query35	4.50	4.49	4.45
query36	0.65	0.49	0.47
query37	0.18	0.16	0.15
query38	0.16	0.14	0.15
query39	0.05	0.04	0.04
query40	0.16	0.13	0.13
query41	0.10	0.04	0.05
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 111.27 s
Total hot run time: 32.41 s

@morningman

morningman commented Sep 1, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Sep 1, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 38130 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9561e150a5d936d3de2e1a483dbabad5937b161b, data reload: false
------ Round 1 ----------------------------------
q1	18763	4654	4330	4330
q2	2037	188	183	183
q3	11226	964	1057	964
q4	10218	752	746	746
q5	7771	2876	2805	2805
q6	226	139	135	135
q7	962	606	590	590
q8	9323	2081	2106	2081
q9	7217	6539	6517	6517
q10	7002	2177	2225	2177
q11	502	244	240	240
q12	404	223	227	223
q13	18305	3056	3009	3009
q14	288	234	240	234
q15	513	504	489	489
q16	572	529	498	498
q17	982	661	681	661
q18	7671	6907	6879	6879
q19	1392	1049	1103	1049
q20	694	342	330	330
q21	3966	3072	2968	2968
q22	1130	1035	1022	1022
Total cold run time: 111164 ms
Total hot run time: 38130 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4362	4342	4277	4277
q2	388	275	265	265
q3	2925	2684	2658	2658
q4	1930	1645	1710	1645
q5	5682	5701	5804	5701
q6	237	133	138	133
q7	2236	1849	1823	1823
q8	3305	3419	3539	3419
q9	8935	8800	8825	8800
q10	3625	3386	3313	3313
q11	593	515	514	514
q12	843	673	676	673
q13	13201	3086	3237	3086
q14	324	286	277	277
q15	529	486	491	486
q16	619	574	578	574
q17	1853	1557	1547	1547
q18	8235	7879	7846	7846
q19	1808	1562	1492	1492
q20	2145	1915	1881	1881
q21	5694	5451	5490	5451
q22	1155	1052	1037	1037
Total cold run time: 70624 ms
Total hot run time: 56898 ms

@doris-robot

doris-robot commented Sep 1, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 193133 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9561e150a5d936d3de2e1a483dbabad5937b161b, data reload: false
query1	1266	899	841	841
query2	6318	1902	1954	1902
query3	10740	4195	4108	4108
query4	59684	25169	23256	23256
query5	5420	492	494	492
query6	412	164	164	164
query7	5779	301	294	294
query8	299	215	215	215
query9	9168	2537	2510	2510
query10	493	278	270	270
query11	18110	15153	15337	15153
query12	165	111	100	100
query13	1558	381	392	381
query14	10929	7430	7463	7430
query15	218	185	170	170
query16	7508	479	477	477
query17	1121	565	565	565
query18	2105	310	293	293
query19	287	156	146	146
query20	116	114	111	111
query21	205	104	103	103
query22	4567	4620	4752	4620
query23	34198	33120	33704	33120
query24	5951	2816	2867	2816
query25	528	374	369	369
query26	678	151	162	151
query27	1773	280	277	277
query28	3891	2124	2097	2097
query29	687	395	402	395
query30	237	147	158	147
query31	948	756	779	756
query32	81	56	61	56
query33	466	292	293	292
query34	876	487	480	480
query35	824	741	715	715
query36	1047	926	965	926
query37	147	93	93	93
query38	3985	3817	3835	3817
query39	1442	1396	1377	1377
query40	199	114	112	112
query41	46	44	45	44
query42	111	95	95	95
query43	517	470	473	470
query44	1114	751	761	751
query45	195	168	168	168
query46	1099	734	740	734
query47	1930	1802	1844	1802
query48	368	288	290	288
query49	750	422	433	422
query50	841	408	421	408
query51	7348	7080	7017	7017
query52	97	89	87	87
query53	253	181	183	181
query54	567	442	452	442
query55	77	76	74	74
query56	278	255	265	255
query57	1218	1110	1089	1089
query58	221	239	235	235
query59	2916	2833	2768	2768
query60	302	271	280	271
query61	123	120	119	119
query62	764	656	642	642
query63	223	184	190	184
query64	2753	673	655	655
query65	3209	3116	3145	3116
query66	697	341	329	329
query67	15515	15262	15117	15117
query68	3006	578	576	576
query69	401	275	305	275
query70	1166	1129	1135	1129
query71	348	275	269	269
query72	6226	4033	4011	4011
query73	774	333	333	333
query74	9040	8982	8939	8939
query75	3363	2690	2682	2682
query76	1499	1162	961	961
query77	521	324	326	324
query78	9667	9096	9071	9071
query79	1030	534	538	534
query80	697	488	512	488
query81	463	240	233	233
query82	245	143	148	143
query83	168	148	151	148
query84	255	76	74	74
query85	678	296	281	281
query86	318	295	264	264
query87	4358	4389	4215	4215
query88	3074	2316	2290	2290
query89	374	287	287	287
query90	1942	189	193	189
query91	125	101	94	94
query92	60	109	52	52
query93	1084	541	540	540
query94	699	301	304	301
query95	329	260	263	260
query96	593	267	270	267
query97	3154	3080	3057	3057
query98	211	209	205	205
query99	1548	1281	1288	1281
Total cold run time: 305374 ms
Total hot run time: 193133 ms

@doris-robot

doris-robot commented Sep 1, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.27 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9561e150a5d936d3de2e1a483dbabad5937b161b, data reload: false
query1	0.04	0.04	0.04
query2	0.08	0.05	0.04
query3	0.23	0.05	0.04
query4	1.68	0.07	0.08
query5	0.50	0.49	0.50
query6	1.15	0.74	0.73
query7	0.02	0.02	0.01
query8	0.06	0.04	0.04
query9	0.56	0.47	0.49
query10	0.56	0.52	0.55
query11	0.16	0.12	0.12
query12	0.15	0.12	0.13
query13	0.63	0.59	0.59
query14	2.05	2.13	2.04
query15	0.83	0.84	0.81
query16	0.37	0.38	0.38
query17	1.06	1.02	0.99
query18	0.21	0.20	0.20
query19	1.94	1.71	1.73
query20	0.02	0.01	0.01
query21	15.40	0.67	0.66
query22	4.09	8.26	1.33
query23	18.30	1.42	1.21
query24	2.15	0.23	0.21
query25	0.15	0.08	0.08
query26	0.27	0.18	0.18
query27	0.07	0.07	0.08
query28	13.24	1.02	1.00
query29	12.61	3.38	3.31
query30	0.24	0.06	0.05
query31	2.87	0.42	0.40
query32	3.25	0.50	0.47
query33	2.96	2.97	3.00
query34	17.09	4.38	4.47
query35	4.47	4.49	4.47
query36	0.67	0.49	0.48
query37	0.17	0.16	0.15
query38	0.16	0.14	0.15
query39	0.05	0.04	0.04
query40	0.16	0.13	0.12
query41	0.09	0.05	0.05
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.86 s
Total hot run time: 31.27 s

kaka11chen
kaka11chen approved these changes Sep 2, 2024
View reviewed changes

@kaka11chenkaka11chen left a comment •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

kaka11chen
kaka11chen approved these changes Sep 2, 2024
View reviewed changes

@kaka11chenkaka11chen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 2, 2024
@github-actions

github-actionsBot commented Sep 2, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Sep 2, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

morningman added a commit that referenced this pull request Sep 2, 2024
@morningman
[feature](metacache) add system table catalog_meta_cache_statistics#…
92752b9
…40155 (#40210)
bp #40155
@morningman
morningman merged commit a4956d6 into apache:masterSep 3, 2024
dataroaring pushed a commit that referenced this pull request Sep 3, 2024
@morningman@dataroaring
[feature](metacache) add system table catalog_meta_cache_statistics (#…
61dc07f
…40155)
Add new system table `catalog_meta_cache_statistics`
```
mysql> select * from information_schema.catalog_meta_cache_statistics;
+--------------+-----------------------------+----------------------+---------------------+
| CATALOG_NAME | CACHE_NAME | METRIC_NAME | METRIC_VALUE |
+--------------+-----------------------------+----------------------+---------------------+
| hms | hive_file_cache | eviction_count | 0 |
| hms | hive_file_cache | hit_ratio | 0.6 |
| hms | hive_file_cache | average_load_penalty | 7.60805755E7 |
| hms | hive_file_cache | estimated_size | 2 |
| hms | hive_file_cache | hit_count | 3 |
| hms | hive_file_cache | read_count | 5 |
| hms | hive_partition_cache | eviction_count | 0 |
| hms | hive_partition_cache | hit_ratio | 1.0 |
| hms | hive_partition_cache | average_load_penalty | 0.0 |
| hms | hive_partition_cache | estimated_size | 0 |
| hms | hive_partition_cache | hit_count | 0 |
| hms | hive_partition_cache | read_count | 0 |
| hms | hive_partition_values_cache | eviction_count | 0 |
| hms | hive_partition_values_cache | hit_ratio | 1.0 |
| hms | hive_partition_values_cache | average_load_penalty | 0.0 |
| hms | hive_partition_values_cache | estimated_size | 0 |
| hms | hive_partition_values_cache | hit_count | 0 |
| hms | hive_partition_values_cache | read_count | 0 |
| hms | hudi_partition_cache | eviction_count | 0 |
| hms | hudi_partition_cache | hit_ratio | 1.0 |
| hms | hudi_partition_cache | average_load_penalty | 0.0 |
| hms | hudi_partition_cache | estimated_size | 0 |
| hms | hudi_partition_cache | hit_count | 0 |
| hms | hudi_partition_cache | read_count | 0 |
| iceberg | iceberg_table_cache | eviction_count | 0 |
| iceberg | iceberg_table_cache | hit_ratio | 0.07407407407407407 |
| iceberg | iceberg_table_cache | average_load_penalty | 1.7263263424E8 |
| iceberg | iceberg_table_cache | estimated_size | 25 |
| iceberg | iceberg_table_cache | hit_count | 2 |
| iceberg | iceberg_table_cache | read_count | 27 |
| iceberg | iceberg_snapshot_cache | eviction_count | 0 |
| iceberg | iceberg_snapshot_cache | hit_ratio | 1.0 |
| iceberg | iceberg_snapshot_cache | average_load_penalty | 0.0 |
| iceberg | iceberg_snapshot_cache | estimated_size | 0 |
| iceberg | iceberg_snapshot_cache | hit_count | 0 |
| iceberg | iceberg_snapshot_cache | read_count | 0 |
+--------------+-----------------------------+----------------------+---------------------+
```
@gavinchougavinchou mentioned this pull request Oct 13, 2024
Closed
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@morningman@HappenLee
[feature](metacache) add system table catalog_meta_cache_statistics (a…
3ce32e7
…pache#40155)
Add new system table `catalog_meta_cache_statistics`
```
mysql> select * from information_schema.catalog_meta_cache_statistics;
+--------------+-----------------------------+----------------------+---------------------+
| CATALOG_NAME | CACHE_NAME | METRIC_NAME | METRIC_VALUE |
+--------------+-----------------------------+----------------------+---------------------+
| hms | hive_file_cache | eviction_count | 0 |
| hms | hive_file_cache | hit_ratio | 0.6 |
| hms | hive_file_cache | average_load_penalty | 7.60805755E7 |
| hms | hive_file_cache | estimated_size | 2 |
| hms | hive_file_cache | hit_count | 3 |
| hms | hive_file_cache | read_count | 5 |
| hms | hive_partition_cache | eviction_count | 0 |
| hms | hive_partition_cache | hit_ratio | 1.0 |
| hms | hive_partition_cache | average_load_penalty | 0.0 |
| hms | hive_partition_cache | estimated_size | 0 |
| hms | hive_partition_cache | hit_count | 0 |
| hms | hive_partition_cache | read_count | 0 |
| hms | hive_partition_values_cache | eviction_count | 0 |
| hms | hive_partition_values_cache | hit_ratio | 1.0 |
| hms | hive_partition_values_cache | average_load_penalty | 0.0 |
| hms | hive_partition_values_cache | estimated_size | 0 |
| hms | hive_partition_values_cache | hit_count | 0 |
| hms | hive_partition_values_cache | read_count | 0 |
| hms | hudi_partition_cache | eviction_count | 0 |
| hms | hudi_partition_cache | hit_ratio | 1.0 |
| hms | hudi_partition_cache | average_load_penalty | 0.0 |
| hms | hudi_partition_cache | estimated_size | 0 |
| hms | hudi_partition_cache | hit_count | 0 |
| hms | hudi_partition_cache | read_count | 0 |
| iceberg | iceberg_table_cache | eviction_count | 0 |
| iceberg | iceberg_table_cache | hit_ratio | 0.07407407407407407 |
| iceberg | iceberg_table_cache | average_load_penalty | 1.7263263424E8 |
| iceberg | iceberg_table_cache | estimated_size | 25 |
| iceberg | iceberg_table_cache | hit_count | 2 |
| iceberg | iceberg_table_cache | read_count | 27 |
| iceberg | iceberg_snapshot_cache | eviction_count | 0 |
| iceberg | iceberg_snapshot_cache | hit_ratio | 1.0 |
| iceberg | iceberg_snapshot_cache | average_load_penalty | 0.0 |
| iceberg | iceberg_snapshot_cache | estimated_size | 0 |
| iceberg | iceberg_snapshot_cache | hit_count | 0 |
| iceberg | iceberg_snapshot_cache | read_count | 0 |
+--------------+-----------------------------+----------------------+---------------------+
```
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actionsgithub-actions[bot]github-actions[bot] left review comments
@kaka11chenkaka11chenkaka11chen approved these changes
@wuwenchiwuwenchiwuwenchi approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.1.6-mergeddev/3.0.2-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@morningman@doris-robot@kaka11chen@wuwenchi@dataroaring