[enhancement](zonemap) add beut for olap string convertor for zonemap by yiguolei · Pull Request #61277 · apache/doris

yiguolei · 2026-03-12T10:41:34Z

What problem does this PR solve?

And there is also a potential bug in zonemap. Should opt.ignore_scale = (field_type != FieldType::OLAP_FIELD_TYPE_DECIMAL);

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2026-03-12T10:41:40Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

yiguolei · 2026-03-12T10:42:41Z

run buildall

doris-robot · 2026-03-12T12:16:21Z

TPC-H: Total hot run time: 28059 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3e3a1359756b982fd8fdfb13ffdcc6ea16be9441, data reload: false
------ Round 1 ----------------------------------
============================================
q1	17640	4518	4352	4352
q2	q3	10708	835	522	522
q4	4721	368	253	253
q5	7932	1223	1002	1002
q6	201	174	146	146
q7	841	858	663	663
q8	10442	1568	1443	1443
q9	6219	4762	4771	4762
q10	6336	1928	1639	1639
q11	486	284	244	244
q12	740	566	477	477
q13	18050	2963	2214	2214
q14	233	235	215	215
q15	944	811	800	800
q16	755	729	725	725
q17	738	888	419	419
q18	6009	5401	5368	5368
q19	1393	991	637	637
q20	517	502	397	397
q21	4725	1966	1518	1518
q22	379	325	263	263
Total cold run time: 100009 ms
Total hot run time: 28059 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4792	4619	4596	4596
q2	q3	3888	4409	3895	3895
q4	888	1218	831	831
q5	4101	4426	4375	4375
q6	184	175	143	143
q7	1782	1665	1526	1526
q8	2500	2745	2554	2554
q9	7674	7585	7467	7467
q10	3731	4018	3594	3594
q11	520	435	429	429
q12	502	611	447	447
q13	2850	3130	2339	2339
q14	284	300	283	283
q15	900	819	820	819
q16	710	758	714	714
q17	1214	1468	1364	1364
q18	7459	7158	6592	6592
q19	1010	959	978	959
q20	2117	2172	2020	2020
q21	4026	3609	3376	3376
q22	456	430	382	382
Total cold run time: 51588 ms
Total hot run time: 48705 ms

doris-robot · 2026-03-12T12:26:24Z

TPC-DS: Total hot run time: 153233 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3e3a1359756b982fd8fdfb13ffdcc6ea16be9441, data reload: false
query5	4346	639	502	502
query6	346	228	217	217
query7	4226	459	264	264
query8	335	244	228	228
query9	8679	2718	2722	2718
query10	480	362	350	350
query11	7226	5868	5672	5672
query12	184	131	124	124
query13	1255	462	373	373
query14	5710	3810	3573	3573
query14_1	2826	2768	2766	2766
query15	206	189	178	178
query16	996	464	460	460
query17	1108	686	582	582
query18	2428	437	343	343
query19	213	204	178	178
query20	138	131	130	130
query21	224	152	127	127
query22	4711	4916	4729	4729
query23	16834	16077	15937	15937
query23_1	16061	15739	15821	15739
query24	7566	1661	1288	1288
query24_1	1250	1267	1261	1261
query25	575	499	433	433
query26	1249	266	153	153
query27	2780	481	289	289
query28	4525	1852	1888	1852
query29	880	590	486	486
query30	321	249	214	214
query31	1358	1319	1235	1235
query32	82	78	72	72
query33	514	343	301	301
query34	909	918	572	572
query35	649	686	618	618
query36	1070	1121	982	982
query37	134	98	91	91
query38	2979	2970	2941	2941
query39	890	841	849	841
query39_1	826	858	876	858
query40	238	157	143	143
query41	70	65	64	64
query42	305	300	299	299
query43	254	258	230	230
query44 query45	201	190	188	188
query46	877	980	622	622
query47	2106	2160	2098	2098
query48	324	335	229	229
query49	649	468	432	432
query50	685	271	210	210
query51	4229	4121	4014	4014
query52	288	294	282	282
query53	290	341	282	282
query54	298	274	276	274
query55	90	91	81	81
query56	339	323	310	310
query57	1399	1347	1304	1304
query58	293	280	267	267
query59	1349	1489	1261	1261
query60	346	338	319	319
query61	148	143	141	141
query62	623	591	553	553
query63	307	275	271	271
query64	5091	1259	1059	1059
query65 query66	1453	472	367	367
query67	16554	16441	16318	16318
query68 query69	384	312	288	288
query70	966	993	959	959
query71	341	309	295	295
query72	2822	2752	2443	2443
query73	561	570	319	319
query74	10043	9876	9760	9760
query75	2839	2751	2452	2452
query76	2278	1027	669	669
query77	370	419	320	320
query78	11241	11341	10640	10640
query79	3071	848	581	581
query80	1715	625	534	534
query81	599	275	236	236
query82	992	147	118	118
query83	349	275	241	241
query84	260	112	102	102
query85	912	483	452	452
query86	501	299	328	299
query87	3216	3116	2972	2972
query88	3632	2670	2648	2648
query89	437	376	347	347
query90	2065	184	184	184
query91	167	166	135	135
query92	81	71	71	71
query93	1476	851	495	495
query94	670	316	283	283
query95	576	415	310	310
query96	648	532	234	234
query97	2466	2520	2403	2403
query98	239	215	214	214
query99	1002	985	931	931
Total cold run time: 237718 ms
Total hot run time: 153233 ms

yiguolei · 2026-03-16T06:10:40Z

run buildall

doris-robot · 2026-03-16T07:46:00Z

TPC-H: Total hot run time: 26925 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a2a6a467642de071f51e728bd2668377dcf180ad, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17580	4385	4300	4300
q2	q3	10641	803	522	522
q4	4675	353	251	251
q5	7554	1256	1019	1019
q6	179	174	146	146
q7	792	841	680	680
q8	9292	1531	1341	1341
q9	4898	4776	4744	4744
q10	6258	1915	1672	1672
q11	450	255	245	245
q12	719	594	480	480
q13	18043	2939	2182	2182
q14	225	245	208	208
q15	q16	726	731	667	667
q17	725	848	456	456
q18	6005	5538	5241	5241
q19	1124	1013	624	624
q20	546	505	377	377
q21	4415	1837	1451	1451
q22	542	389	319	319
Total cold run time: 95389 ms
Total hot run time: 26925 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4856	4614	4591	4591
q2	q3	3892	4387	3827	3827
q4	876	1192	802	802
q5	4102	4413	4393	4393
q6	184	174	144	144
q7	1788	1703	1550	1550
q8	2554	2721	2605	2605
q9	7718	7418	7439	7418
q10	3807	3954	3592	3592
q11	534	443	431	431
q12	509	618	468	468
q13	2789	3115	2343	2343
q14	293	299	291	291
q15	q16	759	828	722	722
q17	1153	1361	1319	1319
q18	7151	6873	6723	6723
q19	925	931	1013	931
q20	2085	2164	1996	1996
q21	4008	3484	3321	3321
q22	590	475	413	413
Total cold run time: 50573 ms
Total hot run time: 47880 ms

doris-robot · 2026-03-16T07:56:44Z

TPC-DS: Total hot run time: 168483 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a2a6a467642de071f51e728bd2668377dcf180ad, data reload: false
query5	4336	639	499	499
query6	328	222	198	198
query7	4212	482	266	266
query8	340	244	219	219
query9	8699	2702	2705	2702
query10	539	402	332	332
query11	6958	5095	4884	4884
query12	194	127	132	127
query13	1279	467	341	341
query14	5741	3759	3478	3478
query14_1	2865	2815	2817	2815
query15	211	192	180	180
query16	970	477	457	457
query17	888	724	629	629
query18	2438	452	346	346
query19	216	214	186	186
query20	132	131	126	126
query21	219	133	110	110
query22	13242	14105	14573	14105
query23	16342	15972	15788	15788
query23_1	15655	15429	15412	15412
query24	7140	1625	1246	1246
query24_1	1256	1270	1236	1236
query25	608	458	407	407
query26	1238	260	145	145
query27	2786	479	293	293
query28	4491	1842	1831	1831
query29	849	571	468	468
query30	296	230	189	189
query31	996	943	868	868
query32	83	67	69	67
query33	510	325	271	271
query34	882	889	513	513
query35	625	680	607	607
query36	1065	1114	997	997
query37	131	89	89	89
query38	2899	2944	2880	2880
query39	858	835	829	829
query39_1	795	794	805	794
query40	226	159	137	137
query41	63	59	65	59
query42	260	261	293	261
query43	236	243	220	220
query44 query45	195	192	187	187
query46	871	983	609	609
query47	2138	2128	2121	2121
query48	320	314	224	224
query49	650	460	376	376
query50	669	279	213	213
query51	4071	4037	4020	4020
query52	264	263	259	259
query53	292	340	284	284
query54	296	277	271	271
query55	97	84	84	84
query56	325	340	315	315
query57	1915	1903	1570	1570
query58	282	279	277	277
query59	2817	2967	2749	2749
query60	342	335	328	328
query61	157	155	151	151
query62	624	583	537	537
query63	308	280	289	280
query64	5117	1257	988	988
query65 query66	1451	464	359	359
query67	24299	24452	24286	24286
query68 query69	399	322	284	284
query70	965	944	942	942
query71	331	318	298	298
query72	2841	2711	2411	2411
query73	533	542	318	318
query74	9592	9588	9396	9396
query75	2862	2745	2511	2511
query76	2297	1027	671	671
query77	365	382	319	319
query78	10982	11224	10455	10455
query79	1141	840	567	567
query80	1376	603	544	544
query81	555	259	225	225
query82	1021	150	117	117
query83	350	259	239	239
query84	298	113	97	97
query85	890	483	436	436
query86	416	311	303	303
query87	3134	3106	3057	3057
query88	3500	2670	2666	2666
query89	425	374	344	344
query90	2010	174	171	171
query91	171	162	134	134
query92	77	73	71	71
query93	980	870	508	508
query94	648	322	300	300
query95	590	408	327	327
query96	637	526	236	236
query97	2514	2503	2422	2422
query98	246	226	220	220
query99	964	987	925	925
Total cold run time: 249175 ms
Total hot run time: 168483 ms

github-actions · 2026-03-16T09:57:06Z

PR approved by at least one committer and no changes requested.

github-actions · 2026-03-16T09:57:09Z

PR approved by anyone and no changes requested.

yiguolei · 2026-03-18T06:00:01Z

skip check_coverage

…apache#61277) And there is also a potential bug in zonemap. Should opt.ignore_scale = (field_type != FieldType::OLAP_FIELD_TYPE_DECIMAL); None - Test  - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason  - Behavior changed: - [ ] No. - [ ] Yes.  - Does this need documentation? - [ ] No. - [ ] Yes.  - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label

### What problem does this PR solve? Related PR: apache#54760, apache#61277, apache#65302 Problem Summary: A zone map bound that fails to parse is unknown, not null -- older versions wrote DOUBLE bounds with digits10+1 precision, so DBL_MAX reads back as infinity and is rejected. Mark such a zone map pass_all so it prunes nothing, instead of failing every scan that loads it. ### Release note None Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TPTGDEAgu5sQVYYvfvyTJz

### What problem does this PR solve? Related PR: apache#54760, apache#61277, apache#65302 Problem Summary: A zone map bound that fails to parse is unknown, not null -- older versions wrote DOUBLE bounds with digits10+1 precision, so DBL_MAX reads back as infinity and is rejected. Mark such a zone map pass_all so it prunes nothing, instead of failing every scan that loads it. Pushed-down MIN/MAX falls back to a normal scan when a segment zone map turns out to be invalid. ### Release note None Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TPTGDEAgu5sQVYYvfvyTJz

f
3e3a135

yiguolei added 7 commits March 15, 2026 10:56

f
2db2c13

f
94f3b71

add new method from zonemap string
d136488

f
7240de2

f
e382d1b

f
f63d942

f
a2a6a46

Gabriel39 approved these changes Mar 16, 2026
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Mar 16, 2026

github-actionsBot added the reviewed label Mar 16, 2026

jacktengg approved these changes Mar 18, 2026
View reviewed changes

yiguolei merged commit b8cd32a into apache:masterMar 18, 2026
28 of 30 checks passed

yiguolei mentioned this pull request Mar 23, 2026
pick memcpy alignement and datetime related pr to 4.1 #61620
Merged
16 tasks

yiguolei added the dev/4.1.0-merged label Mar 24, 2026

Navigation Menu

Uh oh!

[enhancement](zonemap) add beut for olap string convertor for zonemap - #61277

[enhancement](zonemap) add beut for olap string convertor for zonemap#61277
yiguolei merged 8 commits into
apache:masterfrom
yiguolei:f4

yiguolei commented Mar 12, 2026•
edited
Loading

Uh oh!

Thearas commented Mar 12, 2026

Uh oh!

yiguolei commented Mar 12, 2026

Uh oh!

doris-robot commented Mar 12, 2026

Uh oh!

doris-robot commented Mar 12, 2026

Uh oh!

yiguolei commented Mar 16, 2026

Uh oh!

doris-robot commented Mar 16, 2026

Uh oh!

doris-robot commented Mar 16, 2026

Uh oh!

github-actionsBot commented Mar 16, 2026

Uh oh!

github-actionsBot commented Mar 16, 2026

Uh oh!

yiguolei commented Mar 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

yiguolei commented Mar 12, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Mar 12, 2026

Uh oh!

yiguolei commented Mar 12, 2026

Uh oh!

doris-robot commented Mar 12, 2026

Uh oh!

doris-robot commented Mar 12, 2026

Uh oh!

yiguolei commented Mar 16, 2026

Uh oh!

doris-robot commented Mar 16, 2026

Uh oh!

doris-robot commented Mar 16, 2026

Uh oh!

github-actionsBot commented Mar 16, 2026

Uh oh!

github-actionsBot commented Mar 16, 2026

Uh oh!

yiguolei commented Mar 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

yiguolei commented Mar 12, 2026•
edited
Loading