[enhance](paimon) opt count pushdown for paimon and refactor be logic by suxiaogang223 · Pull Request #46911 · apache/doris

suxiaogang223 · 2025-01-13T10:53:42Z

What problem does this PR solve?

Related PR: #44038

Obtain directly from statistics for select count(*) from paimon_table
Refactor TableFormatReader, move same logic of IcebergTableReader, PaimonTableReader and TransactionalHiveReader to TableFormatReader

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-01-13T10:53:49Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

suxiaogang223 · 2025-02-21T07:17:34Z

run buildall

doris-robot · 2025-02-21T07:31:27Z

TeamCity cloud ut coverage result:
Function Coverage: 82.25% (1061/1290)
Line Coverage: 65.79% (17595/26746)
Region Coverage: 65.33% (8670/13271)
Branch Coverage: 55.24% (4674/8462)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ef33aa70f28e0294d2512e30c7eac3eabec8e8f0_ef33aa70f28e0294d2512e30c7eac3eabec8e8f0_cloud/report/index.html

doris-robot · 2025-02-21T08:44:30Z

TPC-H: Total hot run time: 31662 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ef33aa70f28e0294d2512e30c7eac3eabec8e8f0, data reload: false
------ Round 1 ----------------------------------
q1	17624	5347	5167	5167
q2	2053	300	189	189
q3	10387	1312	734	734
q4	10219	1018	537	537
q5	7552	2963	2325	2325
q6	206	170	136	136
q7	947	772	642	642
q8	9321	1363	1208	1208
q9	4947	4529	4534	4529
q10	6852	2332	1873	1873
q11	487	281	253	253
q12	349	360	221	221
q13	17776	3699	3183	3183
q14	246	226	215	215
q15	534	479	458	458
q16	623	627	580	580
q17	638	912	344	344
q18	7280	6274	6029	6029
q19	1543	972	583	583
q20	321	325	194	194
q21	2850	2118	1947	1947
q22	361	335	315	315
Total cold run time: 103116 ms
Total hot run time: 31662 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5236	5203	5231	5203
q2	244	336	247	247
q3	2205	2686	2341	2341
q4	1410	1821	1334	1334
q5	4262	4157	4181	4157
q6	217	167	127	127
q7	1896	1866	1791	1791
q8	2648	2601	2555	2555
q9	7208	7087	7091	7087
q10	3024	3194	2755	2755
q11	581	515	507	507
q12	665	778	602	602
q13	3514	3886	3316	3316
q14	276	295	284	284
q15	496	461	472	461
q16	636	666	622	622
q17	1148	1620	1348	1348
q18	7525	7324	7372	7324
q19	797	839	1038	839
q20	1966	2002	1898	1898
q21	5545	5027	4870	4870
q22	602	592	550	550
Total cold run time: 52101 ms
Total hot run time: 50218 ms

doris-robot · 2025-02-21T08:55:57Z

TPC-DS: Total hot run time: 184542 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ef33aa70f28e0294d2512e30c7eac3eabec8e8f0, data reload: false
query1	1031	381	366	366
query2	6517	1922	1881	1881
query3	6794	211	209	209
query4	26313	23800	23671	23671
query5	4326	683	531	531
query6	313	197	187	187
query7	4608	504	304	304
query8	304	245	237	237
query9	8620	2591	2592	2591
query10	486	304	251	251
query11	15653	15127	15218	15127
query12	156	115	110	110
query13	1664	545	390	390
query14	9741	6217	6624	6217
query15	212	207	179	179
query16	7673	646	459	459
query17	1201	736	588	588
query18	1992	413	310	310
query19	197	220	162	162
query20	123	122	117	117
query21	209	131	111	111
query22	4323	4374	4463	4374
query23	34725	33228	32929	32929
query24	7729	2405	2370	2370
query25	532	443	380	380
query26	801	282	157	157
query27	2123	511	321	321
query28	3870	2415	2409	2409
query29	667	536	425	425
query30	233	190	161	161
query31	912	828	764	764
query32	74	64	61	61
query33	549	394	301	301
query34	768	868	515	515
query35	787	824	743	743
query36	963	989	875	875
query37	119	99	100	99
query38	4353	4147	4164	4147
query39	1438	1571	1388	1388
query40	214	118	110	110
query41	57	51	49	49
query42	122	101	104	101
query43	521	513	472	472
query44	1342	800	774	774
query45	178	173	159	159
query46	879	1051	650	650
query47	1770	1771	1723	1723
query48	394	429	304	304
query49	739	487	413	413
query50	694	748	431	431
query51	4203	4196	4099	4099
query52	107	111	93	93
query53	231	260	198	198
query54	493	499	411	411
query55	86	84	78	78
query56	258	264	248	248
query57	1110	1161	1049	1049
query58	256	238	244	238
query59	2621	2946	2572	2572
query60	288	272	254	254
query61	121	119	119	119
query62	812	735	669	669
query63	241	191	197	191
query64	3554	990	652	652
query65	3264	3149	3153	3149
query66	916	409	308	308
query67	15789	15527	15253	15253
query68	8117	792	515	515
query69	485	295	281	281
query70	1232	1132	1092	1092
query71	464	305	281	281
query72	5765	3593	3683	3593
query73	736	766	356	356
query74	9282	9156	9015	9015
query75	3339	3173	2768	2768
query76	3214	1187	763	763
query77	708	414	296	296
query78	9955	10147	9262	9262
query79	2078	832	634	634
query80	675	545	478	478
query81	493	282	236	236
query82	652	127	96	96
query83	192	167	154	154
query84	288	92	77	77
query85	771	352	312	312
query86	348	321	284	284
query87	4407	4438	4332	4332
query88	2905	2261	2239	2239
query89	413	322	294	294
query90	1910	202	196	196
query91	138	138	109	109
query92	75	61	57	57
query93	1153	1002	590	590
query94	699	410	303	303
query95	354	273	256	256
query96	510	546	281	281
query97	2802	2840	2743	2743
query98	227	208	199	199
query99	1422	1394	1266	1266
Total cold run time: 270168 ms
Total hot run time: 184542 ms

doris-robot · 2025-02-21T09:01:18Z

ClickBench: Total hot run time: 30.93 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ef33aa70f28e0294d2512e30c7eac3eabec8e8f0, data reload: false
query1	0.04	0.04	0.04
query2	0.08	0.03	0.03
query3	0.24	0.06	0.06
query4	1.62	0.10	0.10
query5	0.43	0.41	0.40
query6	1.17	0.66	0.66
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.61	0.52	0.51
query10	0.57	0.59	0.57
query11	0.16	0.11	0.10
query12	0.14	0.11	0.12
query13	0.62	0.60	0.61
query14	2.67	2.69	2.69
query15	0.91	0.86	0.87
query16	0.38	0.38	0.38
query17	1.00	1.00	1.03
query18	0.22	0.19	0.20
query19	1.90	1.96	1.75
query20	0.01	0.01	0.02
query21	15.36	0.90	0.55
query22	0.77	1.12	0.89
query23	14.72	1.38	0.62
query24	7.65	0.96	1.06
query25	0.50	0.32	0.18
query26	0.60	0.16	0.16
query27	0.06	0.05	0.05
query28	9.46	0.86	0.44
query29	12.55	3.89	3.25
query30	0.25	0.09	0.07
query31	2.81	0.58	0.38
query32	3.22	0.54	0.46
query33	2.98	3.01	3.07
query34	15.68	5.15	4.48
query35	4.50	4.52	4.56
query36	0.66	0.53	0.48
query37	0.09	0.07	0.06
query38	0.06	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.14	0.14
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 105.1 s
Total hot run time: 30.93 s

doris-robot · 2025-02-21T09:36:46Z

TeamCity be ut coverage result:
Function Coverage: 44.34% (11701/26392)
Line Coverage: 33.92% (98005/288946)
Region Coverage: 33.00% (50183/152081)
Branch Coverage: 28.65% (25225/88042)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ef33aa70f28e0294d2512e30c7eac3eabec8e8f0_ef33aa70f28e0294d2512e30c7eac3eabec8e8f0/report/index.html

suxiaogang223 · 2025-02-21T14:30:02Z

run buildall

doris-robot · 2025-02-21T14:51:20Z

TeamCity cloud ut coverage result:
Function Coverage: 82.25% (1061/1290)
Line Coverage: 65.79% (17596/26746)
Region Coverage: 65.34% (8671/13271)
Branch Coverage: 55.24% (4674/8462)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0fcde840e4f6632e1428501f1f8fbd37fe6878ba_0fcde840e4f6632e1428501f1f8fbd37fe6878ba_cloud/report/index.html

doris-robot · 2025-02-21T15:46:25Z

TPC-H: Total hot run time: 31565 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0fcde840e4f6632e1428501f1f8fbd37fe6878ba, data reload: false
------ Round 1 ----------------------------------
q1	17577	5403	5085	5085
q2	2043	294	167	167
q3	10421	1233	747	747
q4	10201	1006	516	516
q5	7499	2442	2299	2299
q6	190	169	131	131
q7	894	723	602	602
q8	9306	1333	1179	1179
q9	4853	4630	4679	4630
q10	6833	2300	1868	1868
q11	457	285	257	257
q12	347	356	218	218
q13	17765	3687	3173	3173
q14	245	228	210	210
q15	518	467	471	467
q16	626	618	590	590
q17	550	841	324	324
q18	6822	6137	6364	6137
q19	1215	952	534	534
q20	314	325	182	182
q21	2782	2145	1940	1940
q22	365	331	309	309
Total cold run time: 101823 ms
Total hot run time: 31565 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5103	5137	5129	5129
q2	233	332	240	240
q3	2149	2705	2262	2262
q4	1432	1825	1368	1368
q5	4256	4158	4151	4151
q6	208	164	126	126
q7	1871	1826	1653	1653
q8	2592	2630	2568	2568
q9	7198	7189	7132	7132
q10	3033	3176	2794	2794
q11	580	533	494	494
q12	681	763	606	606
q13	3535	3845	3333	3333
q14	279	301	274	274
q15	508	463	472	463
q16	645	690	629	629
q17	1137	1572	1336	1336
q18	7616	7361	7270	7270
q19	786	828	877	828
q20	1959	2018	1895	1895
q21	5407	5028	4836	4836
q22	607	584	548	548
Total cold run time: 51815 ms
Total hot run time: 49935 ms

doris-robot · 2025-02-21T15:58:27Z

TPC-DS: Total hot run time: 190729 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0fcde840e4f6632e1428501f1f8fbd37fe6878ba, data reload: false
query1	1304	958	920	920
query2	6243	1932	1867	1867
query3	10950	4392	4479	4392
query4	53918	24591	23370	23370
query5	5168	551	490	490
query6	367	183	181	181
query7	5123	494	288	288
query8	318	240	236	236
query9	6413	2579	2583	2579
query10	426	300	270	270
query11	15245	15185	14972	14972
query12	173	114	110	110
query13	1145	534	389	389
query14	10641	6697	6659	6659
query15	203	199	188	188
query16	7110	662	461	461
query17	1079	738	608	608
query18	1530	433	322	322
query19	225	204	185	185
query20	131	129	133	129
query21	213	122	111	111
query22	4573	4283	4410	4283
query23	33945	33352	33257	33257
query24	5757	2426	2455	2426
query25	502	464	391	391
query26	677	276	162	162
query27	1922	487	339	339
query28	2841	2498	2460	2460
query29	565	552	426	426
query30	206	195	156	156
query31	876	891	792	792
query32	76	61	62	61
query33	466	363	294	294
query34	737	858	514	514
query35	824	858	760	760
query36	959	1001	902	902
query37	125	99	74	74
query38	4357	4351	4316	4316
query39	1488	1483	1464	1464
query40	203	112	100	100
query41	50	48	56	48
query42	125	111	102	102
query43	507	523	497	497
query44	1352	800	798	798
query45	181	174	163	163
query46	891	1065	662	662
query47	1836	1894	1796	1796
query48	388	432	322	322
query49	689	505	428	428
query50	714	755	446	446
query51	4242	4310	4261	4261
query52	108	99	95	95
query53	238	257	184	184
query54	479	505	422	422
query55	88	81	83	81
query56	285	282	272	272
query57	1170	1205	1119	1119
query58	242	247	274	247
query59	2792	2810	2789	2789
query60	299	286	264	264
query61	127	119	111	111
query62	725	746	727	727
query63	238	192	196	192
query64	1487	1044	680	680
query65	3257	3158	3135	3135
query66	713	395	319	319
query67	15900	15735	15596	15596
query68	5243	766	517	517
query69	509	297	258	258
query70	1183	1077	1129	1077
query71	422	297	262	262
query72	6022	3616	3703	3616
query73	1040	744	352	352
query74	9062	9149	8938	8938
query75	3202	3238	2695	2695
query76	3753	1175	739	739
query77	563	355	273	273
query78	10243	10187	9270	9270
query79	1951	850	605	605
query80	820	537	454	454
query81	534	285	234	234
query82	398	126	91	91
query83	188	167	158	158
query84	289	97	73	73
query85	811	342	301	301
query86	366	294	281	281
query87	4590	4500	4434	4434
query88	2973	2222	2204	2204
query89	390	316	292	292
query90	1679	191	194	191
query91	139	138	108	108
query92	64	58	55	55
query93	2160	1018	575	575
query94	652	383	302	302
query95	344	268	255	255
query96	492	537	267	267
query97	2776	2845	2729	2729
query98	216	198	209	198
query99	1307	1401	1265	1265
Total cold run time: 292528 ms
Total hot run time: 190729 ms

doris-robot · 2025-02-21T16:03:48Z

ClickBench: Total hot run time: 31.11 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0fcde840e4f6632e1428501f1f8fbd37fe6878ba, data reload: false
query1	0.04	0.04	0.03
query2	0.11	0.05	0.05
query3	0.28	0.05	0.05
query4	1.61	0.07	0.08
query5	0.40	0.50	0.40
query6	1.16	0.65	0.65
query7	0.02	0.02	0.02
query8	0.05	0.05	0.05
query9	0.63	0.52	0.52
query10	0.58	0.58	0.57
query11	0.25	0.13	0.13
query12	0.25	0.13	0.13
query13	0.62	0.61	0.62
query14	2.67	2.71	2.75
query15	0.98	0.90	0.89
query16	0.38	0.36	0.36
query17	1.06	1.04	1.03
query18	0.18	0.20	0.19
query19	1.98	1.85	2.02
query20	0.01	0.02	0.01
query21	15.37	0.97	0.66
query22	0.92	1.05	0.76
query23	14.76	1.58	0.75
query24	5.60	0.55	0.28
query25	0.17	0.10	0.08
query26	0.56	0.22	0.18
query27	0.08	0.09	0.08
query28	11.00	1.13	0.55
query29	12.54	4.13	3.45
query30	0.27	0.08	0.06
query31	2.81	0.60	0.42
query32	3.24	0.59	0.48
query33	3.06	3.07	3.16
query34	16.46	5.06	4.37
query35	4.46	4.45	4.45
query36	0.64	0.50	0.50
query37	0.21	0.18	0.16
query38	0.17	0.15	0.16
query39	0.05	0.04	0.04
query40	0.19	0.15	0.16
query41	0.10	0.06	0.05
query42	0.06	0.05	0.05
query43	0.05	0.05	0.05
Total cold run time: 106.03 s
Total hot run time: 31.11 s

suxiaogang223 · 2025-02-28T05:30:51Z

run buildall

doris-robot · 2025-02-28T05:51:11Z

TeamCity cloud ut coverage result:
Function Coverage: 82.15% (1063/1294)
Line Coverage: 65.69% (17612/26809)
Region Coverage: 65.17% (8682/13323)
Branch Coverage: 55.14% (4686/8498)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b7d3fc21254362faedc2ab9d4684ad7674cbbf64_b7d3fc21254362faedc2ab9d4684ad7674cbbf64_cloud/report/index.html

doris-robot · 2025-02-28T07:34:55Z

TPC-H: Total hot run time: 32084 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b7d3fc21254362faedc2ab9d4684ad7674cbbf64, data reload: false
------ Round 1 ----------------------------------
q1	17684	5419	5165	5165
q2	2068	292	170	170
q3	10559	1267	789	789
q4	10238	1031	553	553
q5	7689	2404	2329	2329
q6	194	167	134	134
q7	917	769	617	617
q8	9310	1299	1126	1126
q9	4830	4763	4831	4763
q10	6830	2316	1892	1892
q11	497	296	272	272
q12	358	364	221	221
q13	17772	3802	3135	3135
q14	226	223	208	208
q15	523	464	463	463
q16	625	611	606	606
q17	599	859	362	362
q18	6997	6247	6183	6183
q19	1940	965	560	560
q20	325	319	198	198
q21	2972	2235	2028	2028
q22	377	344	310	310
Total cold run time: 103530 ms
Total hot run time: 32084 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5300	5214	5330	5214
q2	240	330	230	230
q3	2161	2730	2285	2285
q4	1476	1836	1363	1363
q5	4278	4172	4296	4172
q6	214	162	124	124
q7	1868	1914	1797	1797
q8	2639	2709	2538	2538
q9	7202	7157	7179	7157
q10	3015	3213	2804	2804
q11	576	520	499	499
q12	709	788	620	620
q13	3335	4011	3294	3294
q14	291	305	274	274
q15	498	468	467	467
q16	646	705	642	642
q17	1170	1614	1364	1364
q18	7701	7415	7348	7348
q19	850	913	1037	913
q20	1990	2045	1906	1906
q21	5558	5142	4666	4666
q22	653	596	538	538
Total cold run time: 52370 ms
Total hot run time: 50215 ms

doris-robot · 2025-02-28T07:47:01Z

TPC-DS: Total hot run time: 191048 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b7d3fc21254362faedc2ab9d4684ad7674cbbf64, data reload: false
query1	1299	988	951	951
query2	6202	1956	1905	1905
query3	11007	4475	4522	4475
query4	53749	25217	23491	23491
query5	5205	521	468	468
query6	371	215	189	189
query7	5012	512	287	287
query8	319	243	242	242
query9	6409	2615	2592	2592
query10	426	318	272	272
query11	15169	15247	14987	14987
query12	155	109	106	106
query13	1141	523	397	397
query14	11006	6625	6622	6622
query15	224	214	190	190
query16	7057	636	510	510
query17	1098	735	583	583
query18	1581	399	307	307
query19	194	187	155	155
query20	125	124	124	124
query21	256	126	105	105
query22	4498	4624	4290	4290
query23	34022	33375	33265	33265
query24	5673	2433	2393	2393
query25	447	462	418	418
query26	734	270	151	151
query27	2029	503	337	337
query28	2859	2511	2439	2439
query29	579	540	444	444
query30	211	196	153	153
query31	891	872	816	816
query32	79	61	62	61
query33	465	361	295	295
query34	749	856	500	500
query35	805	833	767	767
query36	955	1003	905	905
query37	127	100	71	71
query38	4188	4336	4244	4244
query39	1499	1447	1468	1447
query40	205	116	104	104
query41	52	50	49	49
query42	121	112	106	106
query43	519	516	497	497
query44	1313	810	816	810
query45	182	175	168	168
query46	888	1057	652	652
query47	1833	1837	1787	1787
query48	390	426	334	334
query49	716	497	435	435
query50	705	748	412	412
query51	4314	4327	4208	4208
query52	109	104	113	104
query53	242	293	185	185
query54	493	499	421	421
query55	86	77	80	77
query56	281	264	271	264
query57	1157	1151	1138	1138
query58	244	240	236	236
query59	2719	2831	2961	2831
query60	278	295	256	256
query61	158	120	115	115
query62	726	739	704	704
query63	226	195	189	189
query64	1809	1070	682	682
query65	3309	3228	3251	3228
query66	704	412	289	289
query67	15698	15387	15150	15150
query68	6153	868	501	501
query69	562	292	262	262
query70	1201	1146	1073	1073
query71	454	286	275	275
query72	5904	3660	3780	3660
query73	1256	734	345	345
query74	9139	9067	9003	9003
query75	3397	3164	2677	2677
query76	3808	1172	750	750
query77	570	376	271	271
query78	9961	9975	9361	9361
query79	2229	817	592	592
query80	653	529	445	445
query81	515	276	242	242
query82	663	129	92	92
query83	172	176	149	149
query84	282	97	73	73
query85	762	352	301	301
query86	411	288	295	288
query87	4530	4359	4542	4359
query88	3548	2230	2192	2192
query89	414	319	282	282
query90	1856	196	196	196
query91	134	150	108	108
query92	72	67	55	55
query93	1721	1020	570	570
query94	698	421	329	329
query95	353	272	257	257
query96	487	577	267	267
query97	3310	3373	3256	3256
query98	226	206	198	198
query99	1427	1384	1234	1234
Total cold run time: 295160 ms
Total hot run time: 191048 ms

doris-robot · 2025-02-28T07:52:23Z

ClickBench: Total hot run time: 31.39 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b7d3fc21254362faedc2ab9d4684ad7674cbbf64, data reload: false
query1	0.03	0.03	0.02
query2	0.11	0.05	0.05
query3	0.28	0.06	0.05
query4	1.59	0.08	0.07
query5	0.57	0.55	0.55
query6	1.20	0.71	0.72
query7	0.02	0.01	0.04
query8	0.05	0.05	0.05
query9	0.62	0.52	0.53
query10	0.60	0.59	0.58
query11	0.25	0.12	0.12
query12	0.25	0.12	0.13
query13	0.63	0.62	0.62
query14	2.67	2.69	2.68
query15	1.00	0.87	0.88
query16	0.38	0.38	0.37
query17	1.03	1.05	1.05
query18	0.18	0.20	0.18
query19	1.94	1.86	2.01
query20	0.02	0.01	0.01
query21	15.36	1.00	0.67
query22	0.92	1.04	0.85
query23	14.67	1.55	0.71
query24	4.88	0.61	0.32
query25	0.16	0.09	0.09
query26	0.55	0.21	0.18
query27	0.09	0.08	0.09
query28	11.02	1.19	0.55
query29	12.54	4.05	3.41
query30	0.29	0.09	0.07
query31	2.81	0.64	0.42
query32	3.23	0.59	0.50
query33	3.05	3.05	3.07
query34	16.76	5.19	4.41
query35	4.45	4.54	4.43
query36	0.64	0.52	0.50
query37	0.20	0.16	0.17
query38	0.17	0.15	0.15
query39	0.05	0.04	0.04
query40	0.20	0.16	0.17
query41	0.11	0.05	0.05
query42	0.07	0.05	0.06
query43	0.05	0.05	0.05
Total cold run time: 105.69 s
Total hot run time: 31.39 s

hello-stephen · 2025-02-28T07:55:41Z

BE UT Coverage Report

Increment line coverage 0.00% (0/132) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	44.98% (12001/26682)
Line Coverage	34.47% (100836/292501)
Region Coverage	33.65% (51640/153471)
Branch Coverage	29.41% (26124/88830)

github-actions · 2025-03-02T14:08:36Z

PR approved by at least one committer and no changes requested.

…apache#46911) ### What problem does this PR solve? Related PR: apache#44038 1. Obtain directly from statistics for select count(*) from paimon_table 2. Refactor TableFormatReader, move same logic of IcebergTableReader, PaimonTableReader and TransactionalHiveReader to TableFormatReader

…apache#46911) Related PR: apache#44038 1. Obtain directly from statistics for select count(*) from paimon_table 2. Refactor TableFormatReader, move same logic of IcebergTableReader, PaimonTableReader and TransactionalHiveReader to TableFormatReader

…tor be logic #46911 (#52060) bp #46911

### What problem does this PR solve? Related PR: #46911 1. Fixed the bug of querying count(*) from paimon table which is `merge on read` 2. For each split of the Paimon table, determine whether count(*) optimization can be performed.

Related PR: apache#46911 1. Fixed the bug of querying count(*) from paimon table which is `merge on read` 2. For each split of the Paimon table, determine whether count(*) optimization can be performed.

suxiaogang223 marked this pull request as draft January 13, 2025 10:53

suxiaogang223 force-pushed the table_count_opt branch from f2b8b1a to 00d24fbCompare January 13, 2025 10:55

suxiaogang223 force-pushed the table_count_opt branch from 1c66856 to 4d1346aCompare January 26, 2025 09:59

suxiaogang223 force-pushed the table_count_opt branch from 4d1346a to 7ea07ceCompare February 11, 2025 09:09

suxiaogang223 added 5 commits February 20, 2025 11:37

opt count for paimon
f95d2c1

impl count opt for paimon jni
1e55325

fix
7f1913d

support raw file with deletion vector
bafbe10

fix paimon_reader
80e7eee

suxiaogang223 force-pushed the table_count_opt branch from 7ea07ce to 80e7eeeCompare February 20, 2025 03:37

suxiaogang223 added 2 commits February 20, 2025 16:53

use cardinality to optimize paimon dv table count(*)
58095b0

can't find symbol isEmpty
11f6662

suxiaogang223 marked this pull request as ready for review February 20, 2025 11:15

suxiaogang223 added 4 commits February 20, 2025 19:33

fix
a709849

fix bug
1884797

fix
243579a

add cases
ef33aa7

fix case
0fcde84

fix jni bug
b7d3fc2

morningman approved these changes Mar 2, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Mar 2, 2025

hubgeter approved these changes Mar 2, 2025
View reviewed changes

morningman merged commit 54bb19f into apache:masterMar 2, 2025

suxiaogang223 deleted the table_count_opt branch March 7, 2025 06:24

suxiaogang223 mentioned this pull request Jun 20, 2025
branch-3.1: [enhance](paimon) opt count pushdown for paimon and refactor be logic (#46911) #52060
Merged

suxiaogang223 mentioned this pull request Jun 23, 2025
[fix](paimon) fix select count(*) from paimon table #51834
Merged
16 tasks

morningman added the dev/3.1.x label Jun 24, 2025

github-actionsBot added the dev/3.1.x-conflict label Jun 24, 2025

morrySnow pushed a commit that referenced this pull request Jun 24, 2025
branch-3.1: [enhance](paimon) opt count pushdown for paimon and refac…
52d7650
…tor be logic #46911 (#52060) bp #46911

morrySnow added dev/3.1.0-merged and removed dev/3.1.x dev/3.1.x-conflict labels Jun 24, 2025

morningman removed the dev/3.0.x-experimental label Jul 4, 2025

Uh oh!

Conversation

suxiaogang223 commented Jan 13, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Jan 13, 2025

Uh oh!

suxiaogang223 commented Feb 21, 2025

Uh oh!

doris-robot commented Feb 21, 2025

Uh oh!

doris-robot commented Feb 21, 2025

Uh oh!

doris-robot commented Feb 21, 2025

Uh oh!

doris-robot commented Feb 21, 2025

Uh oh!

doris-robot commented Feb 21, 2025

Uh oh!

suxiaogang223 commented Feb 21, 2025

Uh oh!

doris-robot commented Feb 21, 2025

Uh oh!

doris-robot commented Feb 21, 2025

Uh oh!

doris-robot commented Feb 21, 2025

Uh oh!

doris-robot commented Feb 21, 2025

Uh oh!

suxiaogang223 commented Feb 28, 2025

Uh oh!

doris-robot commented Feb 28, 2025

Uh oh!

doris-robot commented Feb 28, 2025

Uh oh!

doris-robot commented Feb 28, 2025

Uh oh!

doris-robot commented Feb 28, 2025

Uh oh!

hello-stephen commented Feb 28, 2025

BE UT Coverage Report

Uh oh!

github-actionsBot commented Mar 2, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

suxiaogang223 commented Jan 13, 2025•
edited
Loading