[fix](be) Support TIMESTAMPTZ in TopN runtime predicate by jacktengg · Pull Request #63220 · apache/doris

jacktengg · 2026-05-13T09:59:57Z

When the TopnFilter is
emitted, the BE tries to build a RuntimePredicate, but RuntimePredicate::_init rejected TYPE_TIMESTAMPTZ even though every piece of downstream machinery (ComparisonPredicate, Field ordering, VTopNPred via generic le/ge) already supports it.

Add TYPE_TIMESTAMPTZ to the allow-list so the TopN runtime predicate path works for TYPE_TIMESTAMPTZ.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

When the TopnFilter is emitted, the BE tries to build a RuntimePredicate, but `RuntimePredicate::_init` rejected TYPE_TIMESTAMPTZ even though every piece of downstream machinery (ComparisonPredicate, Field ordering, VTopNPred via generic le/ge) already supports it. Add TYPE_TIMESTAMPTZ to the allow-list so the TopN runtime predicate path works for TYPE_TIMESTAMPTZ.

hello-stephen · 2026-05-13T10:00:03Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

jacktengg · 2026-05-13T10:00:15Z

/review

jacktengg · 2026-05-13T10:24:58Z

run buildall

github-actions

Automated review summary:

No blocking issues found in the actual PR diff. The change extends RuntimePredicate::_init to admit TYPE_TIMESTAMPTZ, and the existing predicate creation path already has TIMESTAMPTZ comparison support, so the implementation matches the stated TopN runtime predicate failure.

Critical checkpoint conclusions:

Goal/test coverage: The PR fixes TIMESTAMPTZ TopN runtime predicate initialization and adds regression cases that assert TOPN OPT: is present, plus ASC/DESC, OFFSET, mixed timezone, and NULL ordering coverage.
Scope/minimality: The code change is minimal and localized to runtime predicate type admission; tests are focused on the affected path.
Concurrency/lifecycle: No new shared state, locking, thread, or lifecycle behavior is introduced.
Config/compatibility: No new configuration, wire/storage format, or rolling-upgrade compatibility concern found.
Parallel paths: TIMESTAMPTZ comparison support already exists in predicate creators and storage predicate/index code, so the runtime predicate allow-list was the missing path.
Data correctness: Added tests cover absolute instant ordering and NULLS FIRST/LAST behavior under TopN runtime predicate generation.
Observability/performance: No new observability requirement; the change enables an existing optimization for an already-supported type.

User focus: No additional user-provided review focus was present.

hello-stephen · 2026-05-13T11:15:34Z

TPC-H: Total hot run time: 29512 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eb7fe8151ca09c0d51b6746baced59e9b1da8b9e, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17620	3874	3835	3835
q2	q3	10704	874	601	601
q4	4666	463	347	347
q5	7461	1327	1141	1141
q6	188	170	141	141
q7	926	945	769	769
q8	9334	1411	1301	1301
q9	5643	5325	5362	5325
q10	6244	2058	1782	1782
q11	451	268	259	259
q12	629	420	292	292
q13	18107	3259	2718	2718
q14	298	283	264	264
q15	q16	900	852	781	781
q17	979	1042	744	744
q18	6510	5642	5593	5593
q19	1144	1196	1058	1058
q20	504	387	260	260
q21	4824	2434	1960	1960
q22	477	409	341	341
Total cold run time: 97609 ms
Total hot run time: 29512 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4799	4689	4990	4689
q2	q3	4638	4790	4210	4210
q4	2167	2230	1413	1413
q5	4997	5094	5272	5094
q6	200	169	145	145
q7	2042	1786	1583	1583
q8	3339	3085	3112	3085
q9	8371	8397	8417	8397
q10	4505	4493	4243	4243
q11	622	440	391	391
q12	702	741	538	538
q13	3264	3626	2924	2924
q14	293	306	279	279
q15	q16	778	945	725	725
q17	1363	1296	1237	1237
q18	7887	7125	7166	7125
q19	1143	1129	1159	1129
q20	2291	2225	1938	1938
q21	6263	5443	4926	4926
q22	550	501	427	427
Total cold run time: 60214 ms
Total hot run time: 54498 ms

hello-stephen · 2026-05-13T12:58:57Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	73.78% (27802/37684)
Line Coverage	57.68% (301394/522510)
Region Coverage	54.84% (251245/458143)
Branch Coverage	56.41% (108800/192858)

hello-stephen · 2026-05-13T14:58:03Z

TPC-H: Total hot run time: 29572 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eb7fe8151ca09c0d51b6746baced59e9b1da8b9e, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17636	3888	3866	3866
q2	q3	10720	870	613	613
q4	4658	452	348	348
q5	7457	1336	1134	1134
q6	216	177	144	144
q7	941	943	754	754
q8	9764	1404	1273	1273
q9	6195	5384	5390	5384
q10	6326	2089	1783	1783
q11	471	271	258	258
q12	693	411	285	285
q13	18208	3681	2716	2716
q14	301	282	262	262
q15	q16	909	880	792	792
q17	1056	950	817	817
q18	6447	5655	5685	5655
q19	1346	1276	1042	1042
q20	518	414	262	262
q21	4607	2278	1881	1881
q22	409	353	303	303
Total cold run time: 98878 ms
Total hot run time: 29572 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4177	4043	4049	4043
q2	q3	4611	4765	4182	4182
q4	2116	2180	1392	1392
q5	4969	4921	5168	4921
q6	191	163	133	133
q7	2021	1975	1752	1752
q8	3470	3158	3152	3152
q9	8462	8446	8330	8330
q10	4492	4511	4247	4247
q11	642	447	422	422
q12	715	744	515	515
q13	3192	3620	2953	2953
q14	293	303	285	285
q15	q16	789	799	698	698
q17	1391	1340	1305	1305
q18	8312	7227	7101	7101
q19	1140	1149	1163	1149
q20	2242	2247	1967	1967
q21	6302	5554	5057	5057
q22	585	518	445	445
Total cold run time: 60112 ms
Total hot run time: 54049 ms

hello-stephen · 2026-05-13T15:08:59Z

TPC-DS: Total hot run time: 170443 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eb7fe8151ca09c0d51b6746baced59e9b1da8b9e, data reload: false
query5	4353	651	504	504
query6	334	232	207	207
query7	4255	537	305	305
query8	321	229	253	229
query9	8834	3997	4004	3997
query10	486	355	302	302
query11	6026	2364	2202	2202
query12	191	130	128	128
query13	1320	618	460	460
query14	6858	5333	5052	5052
query14_1	4369	4347	4351	4347
query15	215	204	182	182
query16	1002	458	443	443
query17	1396	767	642	642
query18	2766	523	372	372
query19	316	210	178	178
query20	138	136	127	127
query21	215	144	121	121
query22	13510	13858	14442	13858
query23	17297	16508	16240	16240
query23_1	16363	16172	16249	16172
query24	7729	1792	1393	1393
query24_1	1381	1354	1326	1326
query25	528	484	440	440
query26	1285	328	163	163
query27	2664	609	315	315
query28	4317	1954	1940	1940
query29	976	630	510	510
query30	300	239	198	198
query31	1121	1056	913	913
query32	89	69	68	68
query33	519	342	274	274
query34	1150	1118	643	643
query35	762	776	683	683
query36	1305	1369	1203	1203
query37	147	99	88	88
query38	3155	3111	3100	3100
query39	917	913	873	873
query39_1	868	872	864	864
query40	237	149	135	135
query41	62	59	60	59
query42	110	106	108	106
query43	317	324	281	281
query44 query45	208	198	192	192
query46	1047	1179	744	744
query47	2317	2310	2215	2215
query48	418	413	291	291
query49	626	538	429	429
query50	708	293	219	219
query51	4333	4232	4184	4184
query52	103	102	89	89
query53	245	264	199	199
query54	317	269	263	263
query55	95	89	81	81
query56	326	304	298	298
query57	1437	1420	1338	1338
query58	292	269	267	267
query59	1518	1611	1399	1399
query60	334	325	315	315
query61	172	174	173	173
query62	672	612	572	572
query63	257	191	207	191
query64	2331	812	690	690
query65 query66	1689	503	433	433
query67	29911	29801	29672	29672
query68 query69	439	334	295	295
query70	998	993	975	975
query71	305	276	263	263
query72	2970	2746	2393	2393
query73	887	790	440	440
query74	5073	4888	4708	4708
query75	2747	2679	2327	2327
query76	2262	1130	741	741
query77	407	433	359	359
query78	12925	12862	12410	12410
query79	1464	952	749	749
query80	1018	568	476	476
query81	499	279	243	243
query82	1336	155	123	123
query83	348	274	250	250
query84	254	139	112	112
query85	921	501	465	465
query86	430	358	323	323
query87	3390	3366	3226	3226
query88	3552	2704	2675	2675
query89	446	381	337	337
query90	1748	183	175	175
query91	205	184	155	155
query92	80	72	72	72
query93	957	937	566	566
query94	637	367	315	315
query95	667	476	357	357
query96	1002	730	317	317
query97	2680	2732	2535	2535
query98	242	239	228	228
query99	1115	1145	984	984
Total cold run time: 254768 ms
Total hot run time: 170443 ms

HappenLee

LGTM

When the TopnFilter is emitted, the BE tries to build a RuntimePredicate, but `RuntimePredicate::_init` rejected TYPE_TIMESTAMPTZ even though every piece of downstream machinery (ComparisonPredicate, Field ordering, VTopNPred via generic le/ge) already supports it. Add TYPE_TIMESTAMPTZ to the allow-list so the TopN runtime predicate path works for TYPE_TIMESTAMPTZ.

…63220 (#63236) Cherry-picked from #63220 Co-authored-by: TengJianPing <tengjianping@selectdb.com>

…63220 (#63237) Cherry-picked from #63220 Co-authored-by: TengJianPing <tengjianping@selectdb.com>

When the TopnFilter is emitted, the BE tries to build a RuntimePredicate, but `RuntimePredicate::_init` rejected TYPE_TIMESTAMPTZ even though every piece of downstream machinery (ComparisonPredicate, Field ordering, VTopNPred via generic le/ge) already supports it. Add TYPE_TIMESTAMPTZ to the allow-list so the TopN runtime predicate path works for TYPE_TIMESTAMPTZ.

jacktengg added dev/4.0.x dev/4.1.x labels May 13, 2026

github-actionsBot reviewed May 13, 2026
View reviewed changes

HappenLee approved these changes May 14, 2026
View reviewed changes

HappenLee merged commit d441f1c into apache:masterMay 14, 2026
35 of 36 checks passed

This was referenced May 14, 2026
branch-4.0: [fix](be) Support TIMESTAMPTZ in TopN runtime predicate #63220 #63236
Merged
branch-4.1: [fix](be) Support TIMESTAMPTZ in TopN runtime predicate #63220 #63237
Merged

morningman pushed a commit that referenced this pull request May 16, 2026
branch-4.0: [fix](be) Support TIMESTAMPTZ in TopN runtime predicate#…
8f91289
…63220 (#63236) Cherry-picked from #63220 Co-authored-by: TengJianPing <tengjianping@selectdb.com>

morningman added dev/4.0.6-merged and removed dev/4.0.x labels May 16, 2026

yiguolei pushed a commit that referenced this pull request May 21, 2026
branch-4.1: [fix](be) Support TIMESTAMPTZ in TopN runtime predicate#…
080a7c4
…63220 (#63237) Cherry-picked from #63220 Co-authored-by: TengJianPing <tengjianping@selectdb.com>

yiguolei added dev/4.1.2-merged and removed dev/4.1.x labels May 21, 2026

morningman mentioned this pull request Jun 3, 2026
4.0.6 Release Notes #64065
Closed

yiguolei mentioned this pull request Jun 14, 2026
4.1.2 Release Notes #64485
Closed

Doris-Breakwater mentioned this pull request Jul 29, 2026
[Bug] doris 4.1 版本无法使用 TIMESTAMPTZ 类型字段去order 排序 #66203
Closed
3 tasks

Navigation Menu

Uh oh!

[fix](be) Support TIMESTAMPTZ in TopN runtime predicate - #63220

[fix](be) Support TIMESTAMPTZ in TopN runtime predicate#63220
HappenLee merged 1 commit into
apache:masterfrom
jacktengg:wt-260513-tz-topn

jacktengg commented May 13, 2026•
edited
Loading

Uh oh!

hello-stephen commented May 13, 2026

Uh oh!

jacktengg commented May 13, 2026

Uh oh!

jacktengg commented May 13, 2026

Uh oh!

github-actionsBot left a comment

Uh oh!

hello-stephen commented May 13, 2026

Uh oh!

hello-stephen commented May 13, 2026

Uh oh!

hello-stephen commented May 13, 2026

Uh oh!

hello-stephen commented May 13, 2026

Uh oh!

HappenLee left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

jacktengg commented May 13, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented May 13, 2026

Uh oh!

jacktengg commented May 13, 2026

Uh oh!

jacktengg commented May 13, 2026

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

hello-stephen commented May 13, 2026

Uh oh!

hello-stephen commented May 13, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented May 13, 2026

Uh oh!

hello-stephen commented May 13, 2026

Uh oh!

HappenLee left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

jacktengg commented May 13, 2026•
edited
Loading