[feature](function) support nth_value window function by zhangstar333 · Pull Request #50559 · apache/doris

zhangstar333 · 2025-04-30T03:57:15Z

What problem does this PR solve?

Problem Summary:
doc: apache/doris-website#2350
support nth_value window function
nth_value(column, offset), offset is positive number

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-04-30T03:57:21Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

zhangstar333 · 2025-04-30T04:01:53Z

run buildall

doris-robot · 2025-04-30T04:53:36Z

TPC-H: Total hot run time: 33702 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9cfbdcdb9836a866fd0b7e7dd4af399eb6666732, data reload: false
------ Round 1 ----------------------------------
q1	26096	5072	5032	5032
q2	2065	271	176	176
q3	10417	1212	683	683
q4	10222	997	536	536
q5	7532	2377	2289	2289
q6	183	161	132	132
q7	913	724	593	593
q8	9331	1237	1087	1087
q9	6758	5078	5105	5078
q10	6866	2305	1869	1869
q11	495	288	270	270
q12	348	356	218	218
q13	17783	3719	3086	3086
q14	223	218	207	207
q15	537	493	484	484
q16	424	415	375	375
q17	591	853	373	373
q18	7471	7079	7041	7041
q19	1914	933	544	544
q20	324	325	216	216
q21	3805	2629	2423	2423
q22	1055	1045	990	990
Total cold run time: 115353 ms
Total hot run time: 33702 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5177	5086	5060	5060
q2	228	317	223	223
q3	2107	2659	2250	2250
q4	1369	1806	1346	1346
q5	4383	4362	4333	4333
q6	218	167	128	128
q7	2014	1870	1761	1761
q8	2598	2688	2564	2564
q9	7260	7199	6985	6985
q10	2989	3168	2751	2751
q11	578	505	479	479
q12	668	743	595	595
q13	3540	3892	3304	3304
q14	276	285	267	267
q15	534	486	486	486
q16	419	481	444	444
q17	1138	1544	1380	1380
q18	7676	7434	7508	7434
q19	801	789	845	789
q20	1949	2033	1951	1951
q21	5082	4744	4749	4744
q22	1105	1063	1054	1054
Total cold run time: 52109 ms
Total hot run time: 50328 ms

doris-robot · 2025-04-30T05:05:49Z

TPC-DS: Total hot run time: 192571 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9cfbdcdb9836a866fd0b7e7dd4af399eb6666732, data reload: false
query1	1393	1096	1079	1079
query2	6227	1752	1746	1746
query3	11128	4416	4680	4416
query4	54063	25348	23062	23062
query5	5055	531	464	464
query6	352	213	190	190
query7	4919	491	292	292
query8	311	252	233	233
query9	5581	2608	2605	2605
query10	431	309	254	254
query11	15309	15552	15251	15251
query12	168	112	101	101
query13	1056	494	385	385
query14	10174	6299	6375	6299
query15	207	187	173	173
query16	7044	643	523	523
query17	1104	758	596	596
query18	1576	402	311	311
query19	193	182	165	165
query20	133	123	126	123
query21	209	129	107	107
query22	4373	4486	4520	4486
query23	34445	33238	33757	33238
query24	6822	2502	2445	2445
query25	440	464	396	396
query26	672	283	150	150
query27	2192	510	342	342
query28	3125	2127	2128	2127
query29	579	601	428	428
query30	274	233	188	188
query31	891	861	760	760
query32	76	69	69	69
query33	457	392	303	303
query34	781	883	531	531
query35	809	875	757	757
query36	925	995	891	891
query37	105	100	76	76
query38	4197	4230	4170	4170
query39	1481	1461	1476	1461
query40	219	119	107	107
query41	58	53	51	51
query42	130	107	117	107
query43	498	534	476	476
query44	1359	824	827	824
query45	186	180	171	171
query46	866	1040	647	647
query47	1837	1861	1780	1780
query48	397	423	307	307
query49	681	497	434	434
query50	673	693	426	426
query51	4304	4228	4196	4196
query52	106	106	99	99
query53	233	270	202	202
query54	600	584	515	515
query55	84	80	83	80
query56	303	316	317	316
query57	1233	1183	1116	1116
query58	285	251	259	251
query59	2579	2682	2511	2511
query60	348	344	335	335
query61	150	166	126	126
query62	723	754	694	694
query63	222	203	192	192
query64	1447	1051	755	755
query65	4401	4254	4248	4248
query66	741	405	296	296
query67	15781	15845	15243	15243
query68	6144	889	517	517
query69	529	299	271	271
query70	1250	1170	1102	1102
query71	433	317	288	288
query72	5737	4765	4732	4732
query73	1233	557	346	346
query74	8909	9257	8892	8892
query75	3228	3252	2687	2687
query76	3850	1182	751	751
query77	530	374	282	282
query78	10140	10085	9376	9376
query79	2862	817	559	559
query80	895	584	439	439
query81	526	254	216	216
query82	714	120	96	96
query83	387	244	226	226
query84	300	96	86	86
query85	791	360	299	299
query86	454	308	276	276
query87	4338	4443	4456	4443
query88	3397	2247	2243	2243
query89	409	325	283	283
query90	1690	213	226	213
query91	140	134	108	108
query92	74	62	56	56
query93	2668	956	575	575
query94	765	419	306	306
query95	357	289	285	285
query96	480	578	280	280
query97	3229	3215	3156	3156
query98	228	206	198	198
query99	1434	1426	1286	1286
Total cold run time: 297842 ms
Total hot run time: 192571 ms

doris-robot · 2025-04-30T05:10:59Z

ClickBench: Total hot run time: 29.03 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9cfbdcdb9836a866fd0b7e7dd4af399eb6666732, data reload: false
query1	0.03	0.04	0.03
query2	0.12	0.10	0.11
query3	0.26	0.19	0.19
query4	1.61	0.19	0.19
query5	0.60	0.58	0.60
query6	1.18	0.72	0.72
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.58	0.52	0.51
query10	0.55	0.59	0.57
query11	0.15	0.10	0.11
query12	0.15	0.11	0.12
query13	0.60	0.60	0.59
query14	0.78	0.80	0.80
query15	0.89	0.85	0.86
query16	0.39	0.38	0.38
query17	1.00	1.04	1.01
query18	0.22	0.20	0.19
query19	1.92	1.77	1.79
query20	0.01	0.01	0.01
query21	15.43	0.91	0.54
query22	0.77	1.20	0.80
query23	14.81	1.37	0.67
query24	7.12	1.63	0.56
query25	0.50	0.29	0.06
query26	0.58	0.16	0.14
query27	0.05	0.05	0.05
query28	9.46	0.87	0.43
query29	12.59	3.99	3.32
query30	0.25	0.09	0.07
query31	2.81	0.59	0.38
query32	3.22	0.55	0.46
query33	3.03	3.01	3.01
query34	15.86	5.19	4.48
query35	4.53	4.58	4.54
query36	0.67	0.49	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.17	0.14	0.12
query41	0.09	0.02	0.03
query42	0.03	0.03	0.02
query43	0.03	0.04	0.03
Total cold run time: 103.27 s
Total hot run time: 29.03 s

hello-stephen · 2025-04-30T05:35:02Z

BE UT Coverage Report

Increment line coverage 5.88% (1/17) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	54.81% (14779/26964)
Line Coverage	43.92% (129675/295230)
Region Coverage	42.62% (66178/155267)
Branch Coverage	37.22% (33398/89740)

zhangstar333 · 2025-04-30T07:04:25Z

run buildall

doris-robot · 2025-04-30T10:29:56Z

BE UT Coverage Report

Increment line coverage 5.88% (1/17) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	54.89% (14794/26953)
Line Coverage	44.00% (129853/295134)
Region Coverage	42.70% (66275/155210)
Branch Coverage	37.22% (33377/89676)

hello-stephen · 2025-04-30T12:55:16Z

BE Regression && UT Coverage Report

Increment line coverage 94.12% (16/17) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	80.19% (21211/26450)
Line Coverage	74.25% (218740/294604)
Region Coverage	72.58% (131779/181576)
Branch Coverage	65.85% (67310/102224)

zhangstar333 · 2025-05-06T03:02:09Z

run buildall

doris-robot · 2025-05-06T03:37:59Z

TPC-H: Total hot run time: 34618 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f97295e693c22a9724a70f65642110772c22371e, data reload: false
------ Round 1 ----------------------------------
q1	26327	5103	5022	5022
q2	2063	284	194	194
q3	10450	1280	751	751
q4	10253	1007	548	548
q5	8320	2349	2393	2349
q6	185	163	134	134
q7	914	755	663	663
q8	9333	1318	1126	1126
q9	6969	5047	5125	5047
q10	6871	2327	1899	1899
q11	477	293	263	263
q12	339	356	213	213
q13	17760	3704	3069	3069
q14	247	242	207	207
q15	552	488	484	484
q16	425	441	373	373
q17	608	863	358	358
q18	7670	7320	7039	7039
q19	1729	966	544	544
q20	331	360	231	231
q21	3986	3445	3148	3148
q22	1016	982	956	956
Total cold run time: 116825 ms
Total hot run time: 34618 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5222	5083	5072	5072
q2	245	327	233	233
q3	2150	2645	2284	2284
q4	1365	1810	1394	1394
q5	4252	4108	4145	4108
q6	204	165	125	125
q7	1910	1795	1645	1645
q8	2485	2470	2477	2470
q9	6805	6747	6755	6747
q10	2878	3091	2638	2638
q11	563	501	478	478
q12	665	710	550	550
q13	3344	3673	3086	3086
q14	268	277	263	263
q15	505	473	471	471
q16	426	475	429	429
q17	1100	1486	1336	1336
q18	7232	7074	7011	7011
q19	786	795	978	795
q20	1914	2032	1818	1818
q21	4984	4658	4745	4658
q22	1060	1011	971	971
Total cold run time: 50363 ms
Total hot run time: 48582 ms

doris-robot · 2025-05-06T03:49:36Z

TPC-DS: Total hot run time: 185549 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f97295e693c22a9724a70f65642110772c22371e, data reload: false
query1	1003	480	503	480
query2	6590	1779	1761	1761
query3	6758	218	214	214
query4	26224	23583	22984	22984
query5	5797	614	461	461
query6	297	196	184	184
query7	4624	495	289	289
query8	304	252	242	242
query9	8636	2537	2562	2537
query10	534	318	264	264
query11	16167	15056	14785	14785
query12	165	110	106	106
query13	1642	493	395	395
query14	10474	6043	6103	6043
query15	198	206	170	170
query16	7613	627	480	480
query17	1444	706	559	559
query18	2012	394	300	300
query19	182	186	150	150
query20	119	114	121	114
query21	221	122	106	106
query22	4153	4298	4060	4060
query23	34004	33057	33005	33005
query24	8087	2371	2376	2371
query25	536	447	386	386
query26	1223	264	154	154
query27	2712	491	324	324
query28	4305	2084	2088	2084
query29	731	572	432	432
query30	287	221	182	182
query31	896	859	753	753
query32	75	70	61	61
query33	565	355	325	325
query34	792	862	522	522
query35	807	808	762	762
query36	946	984	898	898
query37	115	103	81	81
query38	4196	4124	4103	4103
query39	1447	1414	1395	1395
query40	220	122	115	115
query41	64	58	60	58
query42	122	106	116	106
query43	500	495	480	480
query44	1282	796	801	796
query45	179	177	171	171
query46	819	1025	686	686
query47	1785	1796	1746	1746
query48	378	406	298	298
query49	806	526	451	451
query50	624	672	420	420
query51	4049	4151	4064	4064
query52	108	108	102	102
query53	235	255	187	187
query54	598	578	514	514
query55	83	81	79	79
query56	350	301	286	286
query57	1138	1121	1103	1103
query58	262	252	250	250
query59	2574	2709	2656	2656
query60	323	317	309	309
query61	130	126	130	126
query62	776	720	667	667
query63	221	177	188	177
query64	4280	980	714	714
query65	4338	4227	4249	4227
query66	1000	411	305	305
query67	15880	15539	15345	15345
query68	8747	867	508	508
query69	516	302	265	265
query70	1176	1124	1112	1112
query71	509	313	291	291
query72	5278	4689	4669	4669
query73	715	591	339	339
query74	9157	9042	8890	8890
query75	4143	3213	2716	2716
query76	3777	1171	750	750
query77	789	390	294	294
query78	10114	10118	9329	9329
query79	2223	814	561	561
query80	584	563	425	425
query81	479	264	219	219
query82	503	123	97	97
query83	251	250	235	235
query84	252	118	86	86
query85	758	347	317	317
query86	336	277	260	260
query87	4431	4501	4328	4328
query88	3478	2182	2192	2182
query89	462	312	288	288
query90	1949	203	206	203
query91	142	135	111	111
query92	78	60	55	55
query93	1687	943	569	569
query94	658	416	295	295
query95	366	294	280	280
query96	476	556	273	273
query97	3196	3260	3146	3146
query98	239	220	201	201
query99	1658	1400	1295	1295
Total cold run time: 279132 ms
Total hot run time: 185549 ms

doris-robot · 2025-05-06T03:54:47Z

ClickBench: Total hot run time: 29.23 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f97295e693c22a9724a70f65642110772c22371e, data reload: false
query1	0.04	0.04	0.03
query2	0.12	0.10	0.11
query3	0.26	0.19	0.20
query4	1.59	0.19	0.20
query5	0.62	0.61	0.59
query6	1.18	0.72	0.72
query7	0.02	0.02	0.01
query8	0.05	0.03	0.04
query9	0.59	0.53	0.52
query10	0.56	0.57	0.57
query11	0.16	0.10	0.11
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	0.81	0.81	0.81
query15	0.88	0.86	0.86
query16	0.38	0.38	0.40
query17	1.04	1.01	1.05
query18	0.21	0.19	0.19
query19	1.94	1.77	1.87
query20	0.01	0.01	0.01
query21	15.40	0.92	0.54
query22	0.74	1.17	0.64
query23	15.00	1.38	0.58
query24	7.34	1.33	1.00
query25	0.51	0.18	0.08
query26	0.59	0.16	0.13
query27	0.05	0.06	0.06
query28	9.77	0.89	0.46
query29	12.61	3.84	3.22
query30	0.25	0.09	0.07
query31	2.81	0.59	0.39
query32	3.23	0.55	0.46
query33	3.06	3.05	3.02
query34	15.71	5.12	4.48
query35	4.51	4.56	4.51
query36	0.67	0.50	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.95 s
Total hot run time: 29.23 s

hello-stephen · 2025-05-06T04:43:18Z

BE UT Coverage Report

Increment line coverage 5.88% (1/17) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	55.08% (14912/27072)
Line Coverage	44.24% (131183/296550)
Region Coverage	42.89% (66863/155903)
Branch Coverage	37.49% (33738/90004)

hello-stephen · 2025-05-06T06:03:11Z

BE Regression && UT Coverage Report

Increment line coverage 94.12% (16/17) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	80.33% (21344/26570)
Line Coverage	74.38% (220181/296037)
Region Coverage	72.62% (132413/182344)
Branch Coverage	65.94% (67640/102580)

HappenLee

LGTM

github-actions · 2025-05-11T02:14:40Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-05-11T02:14:43Z

PR approved by anyone and no changes requested.

### What problem does this PR solve? Problem Summary: doc: apache/doris-website#2350 support nth_value window function nth_value(column, offset), offset is positive number

### What problem does this PR solve? Related PR: #50559 Problem Summary: nth_value over an upper-bounded/lower-unbounded window frame such as ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING is normalized by reversing the order and frame, then evaluating nth_value with a negative offset over a cumulative frame. The BE nth_value window state replaced the tracked frame row count on each range update, so later rows in the cumulative frame could address the wrong row or return NULL. The fix keeps the cumulative frame row count across range updates, and FE preserves literal offset typing when negating bigint nth_value arguments. This adds BE unit coverage for the reversed cumulative execution path and a regression case comparing nth_value to lead for the original SQL frame. ### Release note Fix nth_value results for ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING window frames.

[feature](function) support nth_value window function
41b036b

update
9cfbdcd

update
62060c0

update
f97295e

HappenLee approved these changes May 11, 2025
View reviewed changes

github-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels May 11, 2025

BiteTheDDDDt approved these changes May 22, 2025
View reviewed changes

zhangstar333 merged commit e6771f8 into apache:masterMay 23, 2025

morrySnow mentioned this pull request Jun 26, 2026
[fix](be) Fix nth_value for upper bounded windows #64864
Merged

Uh oh!

Conversation

zhangstar333 commented Apr 30, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Apr 30, 2025

Uh oh!

zhangstar333 commented Apr 30, 2025

Uh oh!

doris-robot commented Apr 30, 2025

Uh oh!

doris-robot commented Apr 30, 2025

Uh oh!

doris-robot commented Apr 30, 2025

Uh oh!

hello-stephen commented Apr 30, 2025

BE UT Coverage Report

Uh oh!

zhangstar333 commented Apr 30, 2025

Uh oh!

doris-robot commented Apr 30, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Apr 30, 2025

BE Regression && UT Coverage Report

Uh oh!

zhangstar333 commented May 6, 2025

Uh oh!

doris-robot commented May 6, 2025

Uh oh!

doris-robot commented May 6, 2025

Uh oh!

doris-robot commented May 6, 2025

Uh oh!

hello-stephen commented May 6, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented May 6, 2025

BE Regression && UT Coverage Report

Uh oh!

HappenLee left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented May 11, 2025

Uh oh!

github-actionsBot commented May 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

zhangstar333 commented Apr 30, 2025•
edited
Loading