[enchement](orc) improve the read amplification problem caused by orc tiny stripe optimization. by hubgeter · Pull Request #50675 · apache/doris

hubgeter · 2025-05-07T15:20:03Z

What problem does this PR solve?

Related PR: #42004
orc submodule pr: apache/doris-thirdparty#313

Problem Summary:

In the previous PR #42004 , the optimization of orc tiny stripe was introduced. However, if there are many columns in the orc file and only a few of them are used in a query, the tiny stripe optimization will cause serious read amplification.
This PR introduces the orc_tiny_stripe_amplification_factor session variable to adjust the usage scenario of orc tiny stripe optimization. When the actual number of bytes to be read accounts for a larger proportion of the entire stripe than this parameter, read optimization is used. The default value of this parameter is 0.4, and the minimum value is 0.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-05-07T15:20:10Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

hubgeter · 2025-05-07T15:21:49Z

run buildall

hubgeter · 2025-05-16T06:46:40Z

run buildall

doris-robot · 2025-05-16T07:01:34Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.31% (1113/1336)
Line Coverage	66.21% (18662/28187)
Region Coverage	65.76% (9256/14076)
Branch Coverage	55.57% (4981/8964)

doris-robot · 2025-05-16T08:53:00Z

TPC-H: Total hot run time: 33873 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3befcf54004622007a8a703d3830a5115bc115b6, data reload: false
------ Round 1 ----------------------------------
q1	26163	5039	4979	4979
q2	2074	283	188	188
q3	10582	1243	703	703
q4	10249	1013	533	533
q5	7855	2431	2319	2319
q6	188	165	135	135
q7	958	752	608	608
q8	9363	1270	1114	1114
q9	6769	5143	5160	5143
q10	6879	2322	1902	1902
q11	479	295	272	272
q12	352	359	219	219
q13	17776	3679	3082	3082
q14	226	235	217	217
q15	537	486	489	486
q16	429	434	376	376
q17	615	878	379	379
q18	7680	7105	7155	7105
q19	1715	970	576	576
q20	344	342	229	229
q21	4193	3239	2335	2335
q22	1088	984	973	973
Total cold run time: 116514 ms
Total hot run time: 33873 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5160	5064	5104	5064
q2	237	324	233	233
q3	2180	2699	2282	2282
q4	1357	1788	1396	1396
q5	4460	4421	4433	4421
q6	215	170	131	131
q7	1998	1948	1778	1778
q8	2629	2599	2555	2555
q9	7217	7184	6985	6985
q10	3041	3220	2769	2769
q11	585	531	492	492
q12	693	742	656	656
q13	3564	3944	3316	3316
q14	297	308	279	279
q15	514	480	470	470
q16	433	482	446	446
q17	1179	1583	1390	1390
q18	7735	7688	7409	7409
q19	836	881	1044	881
q20	2006	2002	1884	1884
q21	4820	4508	4397	4397
q22	1136	1079	1046	1046
Total cold run time: 52292 ms
Total hot run time: 50280 ms

doris-robot · 2025-05-16T09:05:14Z

TPC-DS: Total hot run time: 193233 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3befcf54004622007a8a703d3830a5115bc115b6, data reload: false
query1	1404	1110	1045	1045
query2	6209	1847	1763	1763
query3	11074	4579	4648	4579
query4	53534	24939	23455	23455
query5	5059	531	462	462
query6	349	227	203	203
query7	4890	520	300	300
query8	330	259	238	238
query9	5846	2724	2711	2711
query10	437	348	276	276
query11	15061	15066	14804	14804
query12	162	118	109	109
query13	1044	530	432	432
query14	10248	6342	6381	6342
query15	211	201	196	196
query16	7094	671	509	509
query17	1101	756	622	622
query18	1604	420	324	324
query19	206	206	181	181
query20	128	125	117	117
query21	211	129	107	107
query22	4308	4410	4246	4246
query23	34375	33423	33502	33423
query24	6654	2447	2434	2434
query25	463	461	435	435
query26	725	279	156	156
query27	2508	538	351	351
query28	2988	2222	2185	2185
query29	576	588	449	449
query30	272	239	186	186
query31	890	888	823	823
query32	77	66	67	66
query33	452	391	304	304
query34	813	871	559	559
query35	805	830	766	766
query36	931	1005	919	919
query37	118	109	82	82
query38	4352	4305	4178	4178
query39	1492	1463	1459	1459
query40	215	123	112	112
query41	57	58	53	53
query42	127	122	117	117
query43	519	532	484	484
query44	1376	872	840	840
query45	189	172	172	172
query46	874	1020	684	684
query47	1842	1858	1780	1780
query48	423	440	348	348
query49	721	524	439	439
query50	692	716	420	420
query51	4252	4250	4247	4247
query52	119	114	107	107
query53	242	270	189	189
query54	603	596	531	531
query55	88	97	86	86
query56	325	340	317	317
query57	1182	1240	1118	1118
query58	278	272	261	261
query59	2724	2758	2651	2651
query60	345	343	349	343
query61	128	126	136	126
query62	718	752	678	678
query63	240	193	201	193
query64	1796	1058	684	684
query65	4467	4290	4260	4260
query66	705	452	311	311
query67	16017	15495	15299	15299
query68	7012	898	522	522
query69	540	309	278	278
query70	1210	1133	1100	1100
query71	506	338	305	305
query72	6070	4838	4890	4838
query73	1468	656	350	350
query74	9010	9121	8823	8823
query75	3899	3190	2701	2701
query76	4178	1200	765	765
query77	663	360	293	293
query78	10029	10001	9448	9448
query79	2722	811	584	584
query80	644	513	540	513
query81	504	252	222	222
query82	449	132	91	91
query83	355	247	232	232
query84	295	99	90	90
query85	802	344	313	313
query86	387	321	293	293
query87	4290	4418	4333	4333
query88	3753	2322	2311	2311
query89	407	314	289	289
query90	1807	202	206	202
query91	140	160	111	111
query92	76	63	54	54
query93	2114	929	576	576
query94	669	405	292	292
query95	368	292	294	292
query96	507	585	285	285
query97	2722	2755	2621	2621
query98	233	208	197	197
query99	1432	1395	1263	1263
Total cold run time: 299526 ms
Total hot run time: 193233 ms

doris-robot · 2025-05-16T09:10:30Z

ClickBench: Total hot run time: 29.83 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3befcf54004622007a8a703d3830a5115bc115b6, data reload: false
query1	0.03	0.03	0.03
query2	0.15	0.10	0.11
query3	0.34	0.20	0.19
query4	1.59	0.19	0.19
query5	0.47	0.45	0.46
query6	1.16	0.66	0.65
query7	0.03	0.02	0.02
query8	0.05	0.07	0.05
query9	0.61	0.51	0.52
query10	0.56	0.58	0.57
query11	0.25	0.13	0.13
query12	0.25	0.14	0.13
query13	0.63	0.62	0.63
query14	0.79	0.81	0.82
query15	0.96	0.86	0.87
query16	0.37	0.36	0.38
query17	1.05	1.08	1.05
query18	0.19	0.18	0.19
query19	2.00	1.83	1.81
query20	0.01	0.01	0.01
query21	15.43	0.99	0.68
query22	0.91	1.03	0.80
query23	14.73	1.53	0.77
query24	5.54	0.58	0.28
query25	0.16	0.10	0.09
query26	0.56	0.22	0.18
query27	0.09	0.09	0.09
query28	11.00	1.17	0.59
query29	12.55	4.09	3.43
query30	0.28	0.09	0.06
query31	2.84	0.64	0.42
query32	3.24	0.60	0.50
query33	3.13	3.16	3.13
query34	16.77	5.19	4.45
query35	4.52	4.47	4.54
query36	0.63	0.50	0.50
query37	0.19	0.16	0.18
query38	0.18	0.16	0.16
query39	0.05	0.04	0.04
query40	0.20	0.16	0.17
query41	0.10	0.05	0.05
query42	0.05	0.05	0.06
query43	0.06	0.06	0.05
Total cold run time: 104.7 s
Total hot run time: 29.83 s

doris-robot · 2025-05-16T10:23:59Z

BE UT Coverage Report

Increment line coverage 0.00% (0/30) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	55.87% (14883/26641)
Line Coverage	44.65% (131874/295380)
Region Coverage	43.76% (66404/151738)
Branch Coverage	38.35% (34018/88696)

hello-stephen · 2025-05-16T12:50:43Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (30/30) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	79.36% (20815/26228)
Line Coverage	72.63% (214538/295401)
Region Coverage	70.81% (126250/178301)
Branch Coverage	64.60% (65457/101332)

morningman · 2025-05-21T07:14:48Z

- range_finder);
+ //The first stripe in the read file is selected to calculate an approximate
+ // read amplification factor using the tiny stripe optimization.
+ std::vector<bool> selectedColumns;


Suggested change
std::vector<bool> selectedColumns;
std::vector<bool> selected_columns;

hubgeter · 2025-05-25T16:33:14Z

run buildall

hello-stephen · 2025-05-25T16:47:47Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.25% (1113/1337)
Line Coverage	66.18% (18679/28224)
Region Coverage	65.85% (9271/14079)
Branch Coverage	55.62% (4987/8966)

doris-robot · 2025-05-25T18:56:18Z

TPC-H: Total hot run time: 33616 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8b5abb496131be0ae248aa41c43f1e7f17cd0076, data reload: false
------ Round 1 ----------------------------------
q1	26514	5043	5009	5009
q2	2100	301	202	202
q3	10459	1220	692	692
q4	10238	975	519	519
q5	8251	2365	2329	2329
q6	196	161	129	129
q7	895	746	601	601
q8	9305	1262	1128	1128
q9	6850	5019	5106	5019
q10	6863	2351	1878	1878
q11	488	289	278	278
q12	339	350	217	217
q13	17791	3656	3092	3092
q14	234	233	215	215
q15	528	474	498	474
q16	440	438	366	366
q17	618	855	357	357
q18	7629	7104	7033	7033
q19	1591	950	557	557
q20	327	332	239	239
q21	3625	3146	2339	2339
q22	1033	1014	943	943
Total cold run time: 116314 ms
Total hot run time: 33616 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5142	5043	5093	5043
q2	261	329	230	230
q3	2137	2665	2290	2290
q4	1343	1761	1374	1374
q5	4423	4359	4399	4359
q6	211	165	125	125
q7	1979	1927	1736	1736
q8	2558	2512	2463	2463
q9	7324	7185	6906	6906
q10	3035	3244	2764	2764
q11	578	496	485	485
q12	667	741	615	615
q13	3499	3832	3227	3227
q14	276	289	290	289
q15	566	497	510	497
q16	453	500	422	422
q17	1125	1471	1384	1384
q18	7849	7578	7440	7440
q19	837	815	983	815
q20	2005	2059	1903	1903
q21	4769	4434	4358	4358
q22	1113	1146	1004	1004
Total cold run time: 52150 ms
Total hot run time: 49729 ms

doris-robot · 2025-05-25T19:08:34Z

TPC-DS: Total hot run time: 192540 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8b5abb496131be0ae248aa41c43f1e7f17cd0076, data reload: false
query1	1400	1099	1041	1041
query2	6057	1790	1779	1779
query3	11002	4491	4456	4456
query4	54118	24873	22866	22866
query5	4834	578	453	453
query6	353	201	192	192
query7	4868	489	293	293
query8	304	253	240	240
query9	5358	2619	2633	2619
query10	442	371	279	279
query11	15055	15062	14823	14823
query12	158	117	108	108
query13	1029	536	412	412
query14	10110	6226	6520	6226
query15	209	203	168	168
query16	7076	633	510	510
query17	1105	752	619	619
query18	1591	423	323	323
query19	210	197	188	188
query20	133	127	126	126
query21	201	136	110	110
query22	4561	4724	4709	4709
query23	34324	33573	33466	33466
query24	6535	2425	2432	2425
query25	480	480	404	404
query26	703	279	160	160
query27	2282	511	350	350
query28	3041	2156	2151	2151
query29	592	587	445	445
query30	276	221	188	188
query31	837	868	809	809
query32	76	69	66	66
query33	441	364	329	329
query34	776	873	534	534
query35	842	838	752	752
query36	948	997	898	898
query37	110	96	79	79
query38	4318	4210	4152	4152
query39	1508	1461	1432	1432
query40	213	122	113	113
query41	70	55	58	55
query42	130	117	116	116
query43	513	495	473	473
query44	1349	847	832	832
query45	183	168	169	168
query46	847	1045	652	652
query47	1890	1903	1851	1851
query48	422	440	330	330
query49	698	518	455	455
query50	669	714	405	405
query51	4272	4293	4170	4170
query52	120	113	105	105
query53	233	260	181	181
query54	603	584	510	510
query55	91	83	84	83
query56	324	298	285	285
query57	1198	1201	1175	1175
query58	269	255	259	255
query59	2668	2808	2653	2653
query60	327	337	325	325
query61	124	124	129	124
query62	765	768	693	693
query63	229	192	196	192
query64	1826	1051	679	679
query65	4313	4236	4282	4236
query66	709	409	310	310
query67	15984	15683	15476	15476
query68	7111	894	542	542
query69	540	308	277	277
query70	1236	1081	1078	1078
query71	511	322	292	292
query72	5489	4823	5145	4823
query73	1513	652	365	365
query74	9119	8799	8850	8799
query75	4013	3192	2666	2666
query76	4309	1184	739	739
query77	740	379	291	291
query78	10188	10073	9430	9430
query79	5070	764	557	557
query80	672	525	445	445
query81	507	252	225	225
query82	570	129	96	96
query83	294	255	233	233
query84	297	102	94	94
query85	757	361	316	316
query86	369	300	279	279
query87	4331	4401	4401	4401
query88	3810	2293	2269	2269
query89	443	312	291	291
query90	1849	206	201	201
query91	147	146	114	114
query92	73	63	61	61
query93	3291	943	584	584
query94	659	410	311	311
query95	365	301	280	280
query96	498	570	283	283
query97	2738	2799	2618	2618
query98	234	212	209	209
query99	1428	1397	1289	1289
Total cold run time: 302473 ms
Total hot run time: 192540 ms

doris-robot · 2025-05-25T19:13:48Z

ClickBench: Total hot run time: 29.62 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8b5abb496131be0ae248aa41c43f1e7f17cd0076, data reload: false
query1	0.03	0.03	0.03
query2	0.15	0.11	0.11
query3	0.36	0.20	0.20
query4	1.60	0.20	0.20
query5	0.46	0.45	0.47
query6	1.17	0.65	0.66
query7	0.02	0.01	0.01
query8	0.06	0.04	0.04
query9	0.62	0.52	0.53
query10	0.59	0.59	0.59
query11	0.26	0.13	0.13
query12	0.25	0.14	0.14
query13	0.64	0.61	0.61
query14	0.80	0.82	0.82
query15	0.96	0.87	0.88
query16	0.38	0.36	0.37
query17	1.04	1.04	1.07
query18	0.19	0.18	0.19
query19	1.98	1.82	1.88
query20	0.01	0.01	0.01
query21	15.42	0.95	0.66
query22	0.92	0.99	0.80
query23	14.70	1.53	0.77
query24	5.53	0.57	0.28
query25	0.16	0.09	0.09
query26	0.55	0.21	0.18
query27	0.09	0.08	0.08
query28	11.01	1.22	0.56
query29	12.55	4.15	3.42
query30	0.28	0.08	0.06
query31	2.82	0.62	0.45
query32	3.22	0.60	0.51
query33	3.04	3.17	3.06
query34	16.37	5.02	4.40
query35	4.45	4.47	4.39
query36	0.63	0.50	0.50
query37	0.19	0.18	0.16
query38	0.16	0.16	0.15
query39	0.05	0.04	0.04
query40	0.20	0.15	0.16
query41	0.10	0.05	0.04
query42	0.06	0.05	0.05
query43	0.05	0.05	0.04
Total cold run time: 104.12 s
Total hot run time: 29.62 s

hubgeter · 2025-05-28T02:22:13Z

run buildall

hello-stephen · 2025-05-28T02:41:00Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.25% (1113/1337)
Line Coverage	66.13% (18665/28224)
Region Coverage	65.79% (9263/14079)
Branch Coverage	55.52% (4978/8966)

doris-robot · 2025-05-28T04:44:32Z

TPC-H: Total hot run time: 33939 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 451614765afc712965034dc04887a2fdbc68a46f, data reload: false
------ Round 1 ----------------------------------
q1	26210	5084	5039	5039
q2	2070	282	192	192
q3	10491	1229	715	715
q4	10226	980	527	527
q5	7663	2385	2317	2317
q6	194	161	132	132
q7	959	733	622	622
q8	9309	1288	1140	1140
q9	6737	5134	5086	5086
q10	6894	2306	1873	1873
q11	496	297	281	281
q12	343	365	210	210
q13	17801	3666	3100	3100
q14	246	224	209	209
q15	533	485	493	485
q16	420	428	381	381
q17	612	863	367	367
q18	7644	7201	7124	7124
q19	1613	962	550	550
q20	330	335	221	221
q21	3785	2547	2402	2402
q22	1071	1014	966	966
Total cold run time: 115647 ms
Total hot run time: 33939 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5181	5138	5099	5099
q2	243	320	226	226
q3	2253	2635	2295	2295
q4	1367	1778	1359	1359
q5	4462	4394	4426	4394
q6	211	168	127	127
q7	1979	1909	1779	1779
q8	2594	2646	2566	2566
q9	7325	7228	6910	6910
q10	3015	3206	2741	2741
q11	590	517	504	504
q12	683	757	628	628
q13	3548	3853	3240	3240
q14	299	303	277	277
q15	506	480	476	476
q16	451	490	449	449
q17	1141	1553	1377	1377
q18	7868	7644	7385	7385
q19	836	908	1031	908
q20	2003	2036	1910	1910
q21	4712	4496	4385	4385
q22	1086	1099	1004	1004
Total cold run time: 52353 ms
Total hot run time: 50039 ms

doris-robot · 2025-05-28T04:56:51Z

TPC-DS: Total hot run time: 192989 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 451614765afc712965034dc04887a2fdbc68a46f, data reload: false
query1	1404	1097	1072	1072
query2	6405	1758	1853	1758
query3	11108	4560	4578	4560
query4	56292	24813	23075	23075
query5	5166	504	475	475
query6	403	229	207	207
query7	5264	519	287	287
query8	333	254	252	252
query9	7036	2627	2637	2627
query10	441	343	282	282
query11	15084	14958	14854	14854
query12	160	113	103	103
query13	1238	537	402	402
query14	10117	6403	6254	6254
query15	207	216	190	190
query16	6963	672	515	515
query17	1124	736	576	576
query18	1502	383	302	302
query19	213	186	165	165
query20	154	129	132	129
query21	203	132	126	126
query22	4358	4470	4306	4306
query23	34239	33538	33697	33538
query24	6662	2463	2471	2463
query25	464	474	410	410
query26	647	283	162	162
query27	2183	538	356	356
query28	2990	2159	2158	2158
query29	579	566	461	461
query30	273	214	195	195
query31	874	844	785	785
query32	74	64	67	64
query33	454	357	320	320
query34	787	893	531	531
query35	813	832	760	760
query36	932	1018	914	914
query37	153	101	77	77
query38	4224	4273	4242	4242
query39	1557	1467	1482	1467
query40	212	122	114	114
query41	59	62	55	55
query42	133	120	113	113
query43	501	529	499	499
query44	1369	847	840	840
query45	176	178	169	169
query46	873	1030	657	657
query47	1808	1889	1804	1804
query48	402	445	344	344
query49	721	510	430	430
query50	689	728	415	415
query51	4188	4286	4167	4167
query52	117	111	99	99
query53	232	272	204	204
query54	607	606	520	520
query55	88	82	88	82
query56	314	314	286	286
query57	1197	1186	1127	1127
query58	275	262	263	262
query59	2635	2739	2636	2636
query60	324	320	324	320
query61	124	119	124	119
query62	758	720	703	703
query63	230	197	193	193
query64	1385	1084	706	706
query65	4300	4278	4221	4221
query66	705	401	306	306
query67	15777	15544	15342	15342
query68	6684	905	518	518
query69	551	315	271	271
query70	1201	1112	1099	1099
query71	509	335	286	286
query72	5965	4882	5005	4882
query73	1233	753	366	366
query74	9305	9080	9189	9080
query75	3776	3168	2696	2696
query76	4320	1194	746	746
query77	604	393	350	350
query78	10186	10108	9276	9276
query79	3719	834	559	559
query80	813	502	451	451
query81	504	253	218	218
query82	487	130	100	100
query83	355	260	237	237
query84	300	105	81	81
query85	796	349	359	349
query86	409	318	283	283
query87	4351	4463	4327	4327
query88	3568	2314	2281	2281
query89	403	319	279	279
query90	1799	213	222	213
query91	146	141	112	112
query92	69	61	60	60
query93	2853	924	584	584
query94	663	407	302	302
query95	372	294	288	288
query96	501	560	282	282
query97	2729	2765	2791	2765
query98	235	203	206	203
query99	1468	1397	1292	1292
Total cold run time: 304153 ms
Total hot run time: 192989 ms

doris-robot · 2025-05-28T05:02:05Z

ClickBench: Total hot run time: 29.71 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 451614765afc712965034dc04887a2fdbc68a46f, data reload: false
query1	0.04	0.03	0.03
query2	0.15	0.12	0.11
query3	0.36	0.19	0.19
query4	1.59	0.19	0.19
query5	0.46	0.47	0.45
query6	1.16	0.67	0.66
query7	0.02	0.02	0.02
query8	0.05	0.05	0.04
query9	0.62	0.52	0.51
query10	0.56	0.59	0.57
query11	0.27	0.15	0.13
query12	0.25	0.14	0.13
query13	0.62	0.61	0.62
query14	0.80	0.83	0.81
query15	0.96	0.87	0.88
query16	0.39	0.39	0.37
query17	1.04	1.06	1.03
query18	0.17	0.18	0.17
query19	2.03	1.85	1.85
query20	0.01	0.01	0.01
query21	15.40	0.96	0.66
query22	0.92	1.00	0.77
query23	14.75	1.54	0.74
query24	5.40	0.55	0.28
query25	0.16	0.08	0.09
query26	0.55	0.21	0.17
query27	0.09	0.08	0.08
query28	11.03	1.16	0.58
query29	12.56	4.08	3.42
query30	0.28	0.09	0.06
query31	2.83	0.62	0.43
query32	3.23	0.60	0.52
query33	3.07	3.08	3.17
query34	16.22	5.08	4.43
query35	4.54	4.55	4.51
query36	0.63	0.50	0.49
query37	0.19	0.17	0.18
query38	0.18	0.17	0.15
query39	0.05	0.04	0.04
query40	0.19	0.15	0.16
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.06	0.05	0.05
Total cold run time: 104.04 s
Total hot run time: 29.71 s

hubgeter · 2025-05-28T10:42:47Z

run buildall

hello-stephen · 2025-05-28T11:00:28Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.25% (1113/1337)
Line Coverage	66.16% (18673/28224)
Region Coverage	65.82% (9267/14079)
Branch Coverage	55.59% (4984/8966)

doris-robot · 2025-05-28T12:55:52Z

TPC-H: Total hot run time: 33607 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 451614765afc712965034dc04887a2fdbc68a46f, data reload: false
------ Round 1 ----------------------------------
q1	26400	5025	4924	4924
q2	2081	279	186	186
q3	10492	1220	684	684
q4	10233	1005	502	502
q5	7606	2702	2416	2416
q6	175	162	132	132
q7	910	722	611	611
q8	9306	1277	1106	1106
q9	6905	5084	5116	5084
q10	6868	2290	1913	1913
q11	490	301	281	281
q12	346	352	212	212
q13	17800	3649	3086	3086
q14	233	222	211	211
q15	551	501	478	478
q16	412	434	382	382
q17	622	855	371	371
q18	7585	7131	6960	6960
q19	1694	982	564	564
q20	334	326	213	213
q21	3730	3079	2337	2337
q22	1034	1024	954	954
Total cold run time: 115807 ms
Total hot run time: 33607 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5140	4989	5311	4989
q2	237	327	233	233
q3	2159	2645	2270	2270
q4	1331	1782	1357	1357
q5	4442	4390	4481	4390
q6	209	172	128	128
q7	2028	1937	1799	1799
q8	2572	2643	2596	2596
q9	7253	7239	7049	7049
q10	3059	3165	2760	2760
q11	597	517	496	496
q12	683	770	585	585
q13	3484	3871	3227	3227
q14	293	303	303	303
q15	523	497	491	491
q16	428	490	445	445
q17	1150	1537	1365	1365
q18	7874	7622	7271	7271
q19	795	765	859	765
q20	2049	2154	1834	1834
q21	4864	4627	4412	4412
q22	1125	1011	1030	1011
Total cold run time: 52295 ms
Total hot run time: 49776 ms

doris-robot · 2025-05-28T13:08:04Z

TPC-DS: Total hot run time: 192368 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 451614765afc712965034dc04887a2fdbc68a46f, data reload: false
query1	1444	1089	1095	1089
query2	6288	1886	1902	1886
query3	11023	4418	4500	4418
query4	53832	25211	22982	22982
query5	5181	518	459	459
query6	332	209	196	196
query7	4879	492	292	292
query8	280	220	219	219
query9	5647	2608	2643	2608
query10	434	352	281	281
query11	15050	14893	14769	14769
query12	170	112	114	112
query13	1063	532	425	425
query14	10215	6335	6335	6335
query15	213	203	190	190
query16	7140	649	512	512
query17	1103	733	609	609
query18	1530	417	312	312
query19	197	211	191	191
query20	131	129	116	116
query21	212	130	115	115
query22	4432	4443	4444	4443
query23	34229	33646	33636	33636
query24	6586	2469	2414	2414
query25	480	468	412	412
query26	719	290	156	156
query27	2411	528	343	343
query28	3067	2184	2167	2167
query29	592	588	468	468
query30	280	221	191	191
query31	871	846	777	777
query32	77	65	60	60
query33	465	368	318	318
query34	786	877	551	551
query35	861	833	739	739
query36	939	999	889	889
query37	122	106	79	79
query38	4165	4303	4167	4167
query39	1538	1436	1460	1436
query40	216	126	117	117
query41	62	58	59	58
query42	126	112	110	110
query43	490	509	505	505
query44	1355	835	843	835
query45	191	175	166	166
query46	865	1051	672	672
query47	1830	1894	1789	1789
query48	406	428	328	328
query49	673	509	399	399
query50	681	716	416	416
query51	4248	4217	4139	4139
query52	118	109	108	108
query53	230	264	198	198
query54	577	581	513	513
query55	84	86	87	86
query56	322	306	283	283
query57	1164	1196	1119	1119
query58	266	282	266	266
query59	2768	2829	2710	2710
query60	336	320	313	313
query61	126	127	154	127
query62	706	715	638	638
query63	224	183	185	183
query64	1831	1015	671	671
query65	4256	4224	4224	4224
query66	742	412	310	310
query67	15834	15603	15308	15308
query68	6996	837	516	516
query69	540	302	268	268
query70	1197	1139	1051	1051
query71	503	327	293	293
query72	5962	4922	4900	4900
query73	1432	644	363	363
query74	9198	9168	8783	8783
query75	3788	3234	2700	2700
query76	4229	1205	757	757
query77	636	364	296	296
query78	10012	10201	9474	9474
query79	3164	815	561	561
query80	622	558	439	439
query81	478	258	217	217
query82	450	129	98	98
query83	344	257	230	230
query84	285	116	83	83
query85	828	345	327	327
query86	369	312	283	283
query87	4536	4367	4321	4321
query88	3161	2234	2241	2234
query89	407	318	292	292
query90	1931	207	205	205
query91	142	141	112	112
query92	73	62	58	58
query93	1651	933	575	575
query94	677	419	314	314
query95	366	287	284	284
query96	488	572	285	285
query97	2727	2786	2641	2641
query98	236	209	196	196
query99	1421	1390	1259	1259
Total cold run time: 298560 ms
Total hot run time: 192368 ms

doris-robot · 2025-05-28T13:13:18Z

ClickBench: Total hot run time: 29.7 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 451614765afc712965034dc04887a2fdbc68a46f, data reload: false
query1	0.03	0.04	0.02
query2	0.14	0.11	0.11
query3	0.35	0.20	0.19
query4	1.59	0.19	0.19
query5	0.47	0.46	0.44
query6	1.16	0.66	0.66
query7	0.02	0.02	0.01
query8	0.06	0.04	0.05
query9	0.62	0.51	0.51
query10	0.59	0.56	0.58
query11	0.25	0.15	0.13
query12	0.25	0.13	0.14
query13	0.64	0.62	0.62
query14	0.80	0.84	0.82
query15	0.98	0.88	0.88
query16	0.38	0.36	0.38
query17	1.09	1.03	1.03
query18	0.18	0.18	0.18
query19	1.96	1.85	1.88
query20	0.02	0.01	0.02
query21	15.42	0.97	0.67
query22	0.93	1.06	0.82
query23	14.68	1.56	0.74
query24	5.26	0.60	0.30
query25	0.18	0.09	0.09
query26	0.56	0.23	0.18
query27	0.09	0.09	0.09
query28	11.06	1.22	0.57
query29	12.59	4.16	3.42
query30	0.29	0.08	0.07
query31	2.82	0.66	0.42
query32	3.25	0.62	0.51
query33	3.07	3.05	3.15
query34	16.36	5.14	4.42
query35	4.42	4.46	4.46
query36	0.64	0.51	0.50
query37	0.20	0.18	0.17
query38	0.17	0.17	0.16
query39	0.05	0.04	0.05
query40	0.18	0.16	0.15
query41	0.11	0.04	0.05
query42	0.05	0.05	0.05
query43	0.06	0.05	0.04
Total cold run time: 104.02 s
Total hot run time: 29.7 s

morningman

LGTM

… tiny stripe optimization.

hubgeter · 2025-05-30T08:58:35Z

run buildall

hello-stephen · 2025-05-30T09:18:21Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.31% (1118/1342)
Line Coverage	66.76% (19135/28663)
Region Coverage	66.41% (9471/14261)
Branch Coverage	56.31% (5128/9106)

doris-robot · 2025-05-30T11:17:55Z

TPC-H: Total hot run time: 33799 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 41221b8902788fa7f5fb52afe4ce169e3e8cdfc9, data reload: false
------ Round 1 ----------------------------------
q1	26368	5067	5040	5040
q2	1965	279	176	176
q3	10437	1261	700	700
q4	10241	1001	548	548
q5	7733	2408	2282	2282
q6	188	164	131	131
q7	909	724	594	594
q8	9308	1242	1080	1080
q9	6851	5034	5088	5034
q10	6893	2343	1913	1913
q11	484	300	274	274
q12	346	352	216	216
q13	17770	3711	3103	3103
q14	247	228	209	209
q15	570	484	500	484
q16	430	443	378	378
q17	607	865	394	394
q18	7716	7228	7126	7126
q19	1646	976	567	567
q20	329	346	223	223
q21	3912	2554	2365	2365
q22	1025	1039	962	962
Total cold run time: 115975 ms
Total hot run time: 33799 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5097	5170	5050	5050
q2	239	324	223	223
q3	2152	2666	2326	2326
q4	1408	1780	1388	1388
q5	4543	4441	4465	4441
q6	223	172	126	126
q7	2059	1963	1808	1808
q8	2596	2559	2582	2559
q9	7230	7127	6954	6954
q10	3039	3215	2730	2730
q11	581	514	494	494
q12	695	802	614	614
q13	3567	3914	3339	3339
q14	289	325	278	278
q15	528	489	483	483
q16	441	486	436	436
q17	1169	1607	1384	1384
q18	7808	7532	7484	7484
q19	815	879	945	879
q20	2023	2081	1919	1919
q21	4993	4677	4584	4584
q22	1104	1075	1030	1030
Total cold run time: 52599 ms
Total hot run time: 50529 ms

doris-robot · 2025-05-30T11:30:06Z

TPC-DS: Total hot run time: 194098 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 41221b8902788fa7f5fb52afe4ce169e3e8cdfc9, data reload: false
query1	1409	1114	1088	1088
query2	6331	1899	1891	1891
query3	11050	4446	4450	4446
query4	52392	25643	23601	23601
query5	5096	533	462	462
query6	358	232	203	203
query7	4952	510	305	305
query8	289	242	217	217
query9	5692	2693	2682	2682
query10	451	333	280	280
query11	15031	15125	15174	15125
query12	159	113	106	106
query13	1075	537	424	424
query14	10237	6427	6498	6427
query15	238	202	190	190
query16	7189	665	504	504
query17	1075	715	588	588
query18	1666	408	302	302
query19	195	228	163	163
query20	128	122	116	116
query21	200	123	104	104
query22	4319	4316	4306	4306
query23	34432	33903	33706	33706
query24	6717	2447	2464	2447
query25	470	498	422	422
query26	684	282	169	169
query27	2215	518	351	351
query28	3060	2169	2170	2169
query29	616	578	439	439
query30	275	215	190	190
query31	876	863	771	771
query32	72	61	67	61
query33	460	357	317	317
query34	794	875	555	555
query35	797	832	726	726
query36	962	1013	882	882
query37	115	98	76	76
query38	4291	4249	4189	4189
query39	1550	1478	1494	1478
query40	213	126	113	113
query41	61	62	58	58
query42	119	116	114	114
query43	507	518	530	518
query44	1367	844	859	844
query45	192	174	177	174
query46	861	1037	663	663
query47	1841	1824	1784	1784
query48	415	452	327	327
query49	657	496	430	430
query50	691	764	421	421
query51	4321	4293	4196	4196
query52	122	118	106	106
query53	240	263	191	191
query54	592	606	517	517
query55	93	87	84	84
query56	299	317	313	313
query57	1142	1184	1111	1111
query58	263	278	272	272
query59	2693	2947	2697	2697
query60	335	334	335	334
query61	125	146	148	146
query62	794	727	695	695
query63	238	201	193	193
query64	1758	1046	698	698
query65	4229	4172	4141	4141
query66	712	398	300	300
query67	16056	15604	15572	15572
query68	5629	903	527	527
query69	530	305	273	273
query70	1230	1108	1088	1088
query71	440	311	331	311
query72	5956	4854	5086	4854
query73	1322	718	359	359
query74	9322	8858	9127	8858
query75	3247	3212	2714	2714
query76	3739	1190	731	731
query77	547	380	311	311
query78	10148	10288	9391	9391
query79	1938	833	574	574
query80	670	530	449	449
query81	495	259	217	217
query82	383	127	103	103
query83	249	285	242	242
query84	294	109	90	90
query85	752	360	313	313
query86	320	294	271	271
query87	4436	4371	4387	4371
query88	2930	2352	2335	2335
query89	395	312	287	287
query90	1694	203	210	203
query91	143	139	113	113
query92	65	63	60	60
query93	1885	950	583	583
query94	648	407	295	295
query95	374	293	290	290
query96	486	582	291	291
query97	2731	2743	2641	2641
query98	231	226	202	202
query99	1319	1391	1261	1261
Total cold run time: 293402 ms
Total hot run time: 194098 ms

doris-robot · 2025-05-30T11:35:29Z

ClickBench: Total hot run time: 29.86 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 41221b8902788fa7f5fb52afe4ce169e3e8cdfc9, data reload: false
query1	0.04	0.04	0.03
query2	0.14	0.11	0.10
query3	0.35	0.20	0.20
query4	1.59	0.20	0.21
query5	0.47	0.46	0.46
query6	1.16	0.65	0.65
query7	0.02	0.02	0.02
query8	0.06	0.05	0.04
query9	0.62	0.52	0.54
query10	0.57	0.58	0.57
query11	0.25	0.13	0.14
query12	0.25	0.14	0.13
query13	0.63	0.62	0.64
query14	0.80	0.82	0.83
query15	0.96	0.88	0.87
query16	0.38	0.38	0.37
query17	1.09	1.06	1.02
query18	0.19	0.18	0.19
query19	2.05	1.86	1.87
query20	0.02	0.01	0.01
query21	15.43	0.98	0.67
query22	0.93	1.00	0.84
query23	14.68	1.54	0.77
query24	5.31	0.59	0.30
query25	0.17	0.09	0.08
query26	0.56	0.23	0.19
query27	0.09	0.08	0.08
query28	11.00	1.22	0.59
query29	12.55	4.19	3.45
query30	0.28	0.08	0.07
query31	2.82	0.61	0.43
query32	3.23	0.60	0.50
query33	3.04	3.13	3.09
query34	16.75	5.13	4.38
query35	4.49	4.44	4.49
query36	0.64	0.50	0.50
query37	0.20	0.17	0.18
query38	0.18	0.16	0.15
query39	0.05	0.04	0.04
query40	0.19	0.16	0.16
query41	0.11	0.05	0.05
query42	0.07	0.06	0.06
query43	0.06	0.05	0.05
Total cold run time: 104.47 s
Total hot run time: 29.86 s

kaka11chen

LGTM

github-actions · 2025-06-11T02:28:59Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-06-11T02:29:02Z

PR approved by anyone and no changes requested.

github-actions · 2025-12-09T00:14:16Z

We're closing this PR because it hasn't been updated in a while.
This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag!

hubgeter force-pushed the orc_tiny_stripe_enc branch 2 times, most recently from 2801c8d to 3befcf5Compare May 16, 2025 06:46

morningman added dev/2.1.x dev/3.0.x labels May 21, 2025

morningman reviewed May 24, 2025
View reviewed changes

hubgeter force-pushed the orc_tiny_stripe_enc branch from 3befcf5 to c3d4c9bCompare May 25, 2025 16:22

morningman previously approved these changes May 30, 2025
View reviewed changes

[enchement](orc) improve the read amplification problem caused by orc…
41221b8
… tiny stripe optimization.

hubgeter dismissed morningman’s stale review via 41221b8May 30, 2025 08:56

hubgeter force-pushed the orc_tiny_stripe_enc branch from 4516147 to 41221b8Compare May 30, 2025 08:56

kaka11chen approved these changes Jun 11, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jun 11, 2025

github-actionsBot added the reviewed label Jun 11, 2025

github-actionsBot added the Stale label Dec 9, 2025

github-actionsBot closed this Dec 19, 2025

	std::vector<bool> selectedColumns;
	std::vector<bool> selected_columns;

Uh oh!

Conversation

hubgeter commented May 7, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented May 7, 2025

Uh oh!

hubgeter commented May 7, 2025

Uh oh!

hubgeter commented May 16, 2025

Uh oh!

doris-robot commented May 16, 2025

Cloud UT Coverage Report

Uh oh!

doris-robot commented May 16, 2025

Uh oh!

doris-robot commented May 16, 2025

Uh oh!

doris-robot commented May 16, 2025

Uh oh!

doris-robot commented May 16, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented May 16, 2025

BE Regression && UT Coverage Report

Uh oh!

morningmanMay 21, 2025

Choose a reason for hiding this comment

Uh oh!

hubgeter commented May 25, 2025

Uh oh!

hello-stephen commented May 25, 2025

Cloud UT Coverage Report

Uh oh!

doris-robot commented May 25, 2025

Uh oh!

doris-robot commented May 25, 2025

Uh oh!

doris-robot commented May 25, 2025

Uh oh!

hubgeter commented May 28, 2025

Uh oh!

hello-stephen commented May 28, 2025

Cloud UT Coverage Report

Uh oh!

doris-robot commented May 28, 2025

Uh oh!

doris-robot commented May 28, 2025

Uh oh!

doris-robot commented May 28, 2025

Uh oh!

hubgeter commented May 28, 2025

Uh oh!

hello-stephen commented May 28, 2025

Cloud UT Coverage Report

Uh oh!

doris-robot commented May 28, 2025

Uh oh!

doris-robot commented May 28, 2025

Uh oh!

doris-robot commented May 28, 2025

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

hubgeter commented May 30, 2025

Uh oh!

hello-stephen commented May 30, 2025

Cloud UT Coverage Report

Uh oh!

doris-robot commented May 30, 2025

Uh oh!

doris-robot commented May 30, 2025

Uh oh!

doris-robot commented May 30, 2025

Uh oh!

hubgeter commented May 7, 2025•
edited
Loading