[BugFix](PreparedStatement) fix stmtId overflow by eldenmoon · Pull Request #47950 · apache/doris

eldenmoon · 2025-02-16T14:49:46Z

if stmtId larger than Interger.MAX_VALUE, there is chance to return Not supported such prepared statement to the client which indicates the stmtId not exist.In order to fix this bug:

change id from original stmtId to preparedStmtId in ConnectContext, id which increment in session level and only increment when comming new prepared statement
rolling the id to fit the range [-2147483648, 2147483647)

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

if stmtId larger than Interger.MAX_VALUE, there is chance to return `Not supported such prepared statement` to the client which indicates the stmtId not exist.In order to fix this bug: 1. change id from original `stmtId` to `preparedStmtId` in ConnectContext 2. rolling the id to fit the range [0, 2147483647)

eldenmoon · 2025-02-16T14:50:02Z

run buildall

Thearas · 2025-02-16T14:50:08Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

doris-robot · 2025-02-16T15:35:39Z

TPC-H: Total hot run time: 31594 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 41f7ab613d87311442860ccab5b1ce38640b7ddd, data reload: false
------ Round 1 ----------------------------------
q1	17625	5233	5087	5087
q2	2055	307	163	163
q3	10395	1236	757	757
q4	10208	1005	531	531
q5	7518	2361	2381	2361
q6	187	167	137	137
q7	897	741	599	599
q8	9295	1285	1077	1077
q9	4806	4638	4814	4638
q10	6813	2294	1890	1890
q11	466	281	250	250
q12	345	354	222	222
q13	17782	3668	3047	3047
q14	232	225	204	204
q15	508	474	479	474
q16	626	615	577	577
q17	571	870	355	355
q18	6564	6244	6222	6222
q19	1224	960	539	539
q20	307	322	191	191
q21	2739	2245	1970	1970
q22	366	332	303	303
Total cold run time: 101529 ms
Total hot run time: 31594 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5169	5169	5189	5169
q2	235	320	222	222
q3	2154	2668	2300	2300
q4	1408	1802	1396	1396
q5	4237	4148	4157	4148
q6	204	165	124	124
q7	1850	1809	1654	1654
q8	2581	2711	2594	2594
q9	7155	7163	7178	7163
q10	3002	3217	2755	2755
q11	554	526	492	492
q12	716	766	597	597
q13	3557	3848	3346	3346
q14	271	287	269	269
q15	514	470	458	458
q16	636	677	625	625
q17	1137	1586	1348	1348
q18	7638	7308	7365	7308
q19	817	797	815	797
q20	1975	2007	1920	1920
q21	5437	4998	4786	4786
q22	619	588	556	556
Total cold run time: 51866 ms
Total hot run time: 50027 ms

doris-robot · 2025-02-16T15:47:40Z

TPC-DS: Total hot run time: 189904 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 41f7ab613d87311442860ccab5b1ce38640b7ddd, data reload: false
query1	1313	963	949	949
query2	6222	1814	1813	1813
query3	11031	4618	4509	4509
query4	54285	25510	23493	23493
query5	5023	576	484	484
query6	344	190	187	187
query7	4898	500	284	284
query8	313	248	235	235
query9	5341	2497	2493	2493
query10	426	310	258	258
query11	15143	14955	14863	14863
query12	155	108	107	107
query13	1027	515	385	385
query14	10185	6540	6343	6343
query15	207	207	176	176
query16	7121	647	466	466
query17	1045	692	568	568
query18	1587	419	312	312
query19	219	201	163	163
query20	118	125	125	125
query21	218	132	104	104
query22	4351	4446	4237	4237
query23	33704	33403	33455	33403
query24	6606	2446	2404	2404
query25	437	471	401	401
query26	718	272	150	150
query27	2453	503	334	334
query28	2991	2441	2411	2411
query29	572	568	437	437
query30	219	193	161	161
query31	856	848	824	824
query32	76	61	65	61
query33	432	366	311	311
query34	753	866	509	509
query35	808	833	745	745
query36	943	1020	915	915
query37	127	103	76	76
query38	4269	4264	4164	4164
query39	1479	1480	1425	1425
query40	214	131	105	105
query41	58	56	54	54
query42	121	106	104	104
query43	500	510	483	483
query44	1300	802	804	802
query45	182	207	171	171
query46	871	1055	664	664
query47	1869	1843	1800	1800
query48	390	413	313	313
query49	692	533	430	430
query50	720	755	426	426
query51	4301	4313	4276	4276
query52	111	106	100	100
query53	236	260	191	191
query54	490	481	409	409
query55	88	80	83	80
query56	260	273	252	252
query57	1128	1164	1118	1118
query58	239	246	230	230
query59	2789	2863	2781	2781
query60	293	283	280	280
query61	116	119	117	117
query62	731	729	688	688
query63	238	191	192	191
query64	1774	1026	678	678
query65	3310	3111	3125	3111
query66	761	481	293	293
query67	15922	15744	15394	15394
query68	5486	769	496	496
query69	510	283	260	260
query70	1243	1102	1121	1102
query71	432	292	255	255
query72	5983	3636	3762	3636
query73	1400	747	342	342
query74	8890	9249	8989	8989
query75	3151	3116	2675	2675
query76	3826	1160	741	741
query77	538	380	280	280
query78	9920	10041	9337	9337
query79	2423	795	598	598
query80	640	518	441	441
query81	506	276	235	235
query82	251	123	100	100
query83	184	166	148	148
query84	297	88	68	68
query85	772	344	296	296
query86	368	301	280	280
query87	4441	4539	4423	4423
query88	3524	2145	2189	2145
query89	402	315	283	283
query90	1737	190	186	186
query91	138	141	109	109
query92	75	68	56	56
query93	1900	988	562	562
query94	679	379	300	300
query95	342	266	261	261
query96	476	550	263	263
query97	2856	2814	2718	2718
query98	225	206	200	200
query99	1300	1400	1293	1293
Total cold run time: 292634 ms
Total hot run time: 189904 ms

doris-robot · 2025-02-16T15:52:59Z

ClickBench: Total hot run time: 30.4 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 41f7ab613d87311442860ccab5b1ce38640b7ddd, data reload: false
query1	0.03	0.03	0.05
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.62	0.10	0.10
query5	0.41	0.42	0.39
query6	1.19	0.68	0.66
query7	0.02	0.01	0.01
query8	0.04	0.03	0.04
query9	0.58	0.50	0.51
query10	0.58	0.58	0.57
query11	0.15	0.10	0.11
query12	0.14	0.10	0.11
query13	0.60	0.59	0.60
query14	2.69	2.73	2.73
query15	0.92	0.85	0.86
query16	0.37	0.37	0.37
query17	1.05	1.01	1.03
query18	0.22	0.20	0.20
query19	1.87	1.75	1.90
query20	0.01	0.01	0.01
query21	15.36	0.96	0.56
query22	0.76	1.21	0.77
query23	14.81	1.40	0.63
query24	11.64	1.12	0.52
query25	0.40	0.22	0.12
query26	1.00	0.19	0.15
query27	0.06	0.05	0.04
query28	5.80	0.76	0.44
query29	12.57	4.00	3.32
query30	0.25	0.08	0.06
query31	2.82	0.58	0.38
query32	3.22	0.54	0.47
query33	3.05	3.00	3.03
query34	15.83	5.12	4.50
query35	4.62	4.55	4.59
query36	0.64	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.14	0.14
query41	0.07	0.02	0.03
query42	0.03	0.02	0.02
query43	0.03	0.03	0.02
Total cold run time: 106.09 s
Total hot run time: 30.4 s

eldenmoon · 2025-02-17T02:56:41Z

run buildall

qidaye

LGTM

github-actions · 2025-02-17T03:10:42Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-02-17T03:10:45Z

PR approved by anyone and no changes requested.

doris-robot · 2025-02-17T03:18:25Z

TPC-H: Total hot run time: 31770 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 056096014d6361da82ba982ffcc0d37043d21adf, data reload: false
------ Round 1 ----------------------------------
q1	17580	5177	5072	5072
q2	2051	308	167	167
q3	10395	1291	699	699
q4	10214	994	528	528
q5	7480	2410	2426	2410
q6	184	178	134	134
q7	915	759	604	604
q8	9319	1289	1117	1117
q9	4957	4703	4891	4703
q10	6833	2298	1883	1883
q11	468	273	246	246
q12	347	349	211	211
q13	17776	3648	3060	3060
q14	226	229	210	210
q15	520	483	455	455
q16	619	606	573	573
q17	573	868	349	349
q18	6720	6395	6366	6366
q19	1077	971	539	539
q20	310	337	193	193
q21	2780	2133	1954	1954
q22	368	333	297	297
Total cold run time: 101712 ms
Total hot run time: 31770 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5088	5128	5130	5128
q2	237	336	229	229
q3	2163	2708	2298	2298
q4	1419	1942	1376	1376
q5	4230	4149	4335	4149
q6	206	163	122	122
q7	1875	1826	1690	1690
q8	2591	2683	2551	2551
q9	7339	7087	7185	7087
q10	2963	3219	2720	2720
q11	574	496	493	493
q12	671	774	628	628
q13	3481	3823	3239	3239
q14	294	286	276	276
q15	515	453	475	453
q16	628	672	630	630
q17	1120	1577	1346	1346
q18	7614	7443	7285	7285
q19	787	788	874	788
q20	1928	2055	1856	1856
q21	5426	4890	4708	4708
q22	614	579	525	525
Total cold run time: 51763 ms
Total hot run time: 49577 ms

doris-robot · 2025-02-17T03:29:40Z

TPC-DS: Total hot run time: 182901 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 056096014d6361da82ba982ffcc0d37043d21adf, data reload: false
query1	982	371	402	371
query2	6521	1841	1830	1830
query3	6802	214	211	211
query4	26444	23686	22994	22994
query5	4344	681	453	453
query6	282	200	170	170
query7	4620	498	294	294
query8	293	233	216	216
query9	8597	2473	2486	2473
query10	469	313	256	256
query11	15605	15087	14804	14804
query12	146	103	104	103
query13	1673	512	390	390
query14	9380	6527	6117	6117
query15	214	190	181	181
query16	7080	647	492	492
query17	1203	717	561	561
query18	1926	408	318	318
query19	196	188	163	163
query20	123	123	119	119
query21	215	127	104	104
query22	4323	4467	4331	4331
query23	34548	33345	32909	32909
query24	7707	2362	2356	2356
query25	556	457	385	385
query26	1221	267	147	147
query27	2231	500	329	329
query28	3982	2379	2357	2357
query29	688	539	404	404
query30	224	181	154	154
query31	931	850	817	817
query32	72	59	60	59
query33	554	355	303	303
query34	782	839	484	484
query35	788	817	720	720
query36	976	968	876	876
query37	112	101	77	77
query38	4191	4258	4088	4088
query39	1447	1388	1398	1388
query40	208	113	101	101
query41	79	49	48	48
query42	119	104	109	104
query43	480	523	453	453
query44	1245	784	787	784
query45	174	171	156	156
query46	862	1031	640	640
query47	1778	1823	1747	1747
query48	370	397	313	313
query49	768	515	412	412
query50	674	737	411	411
query51	4208	4195	4150	4150
query52	107	103	97	97
query53	235	260	186	186
query54	490	478	395	395
query55	85	84	77	77
query56	279	270	249	249
query57	1109	1144	1059	1059
query58	235	231	232	231
query59	2668	2844	2847	2844
query60	280	266	262	262
query61	120	142	115	115
query62	795	732	669	669
query63	218	191	188	188
query64	4378	980	640	640
query65	3217	3145	3145	3145
query66	1130	405	309	309
query67	16055	15620	15438	15438
query68	2902	790	528	528
query69	457	297	275	275
query70	1213	1031	1102	1031
query71	391	289	257	257
query72	5684	3601	3715	3601
query73	708	738	345	345
query74	8879	8904	8953	8904
query75	3123	3139	2681	2681
query76	3066	1160	779	779
query77	468	370	274	274
query78	9981	10179	9221	9221
query79	969	866	584	584
query80	1054	516	433	433
query81	541	270	235	235
query82	412	132	102	102
query83	191	170	153	153
query84	234	89	81	81
query85	822	343	309	309
query86	375	300	276	276
query87	4486	4423	4386	4386
query88	2789	2156	2175	2156
query89	382	308	285	285
query90	1745	192	190	190
query91	141	135	109	109
query92	64	59	53	53
query93	1112	999	574	574
query94	657	411	298	298
query95	345	261	254	254
query96	478	551	262	262
query97	2779	2806	2756	2756
query98	229	200	209	200
query99	1274	1436	1282	1282
Total cold run time: 262765 ms
Total hot run time: 182901 ms

doris-robot · 2025-02-17T03:34:57Z

ClickBench: Total hot run time: 30.28 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 056096014d6361da82ba982ffcc0d37043d21adf, data reload: false
query1	0.03	0.03	0.04
query2	0.07	0.03	0.04
query3	0.23	0.07	0.06
query4	1.62	0.11	0.10
query5	0.42	0.41	0.39
query6	1.13	0.67	0.64
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.51	0.52
query10	0.58	0.58	0.56
query11	0.15	0.10	0.10
query12	0.15	0.11	0.11
query13	0.61	0.60	0.60
query14	2.72	2.70	2.72
query15	0.92	0.85	0.85
query16	0.38	0.38	0.37
query17	1.00	1.00	0.99
query18	0.21	0.20	0.19
query19	1.91	1.80	1.96
query20	0.01	0.01	0.01
query21	15.36	0.89	0.54
query22	0.74	1.32	0.67
query23	14.80	1.38	0.66
query24	7.99	3.81	0.50
query25	0.30	0.21	0.14
query26	0.80	0.18	0.14
query27	0.05	0.05	0.05
query28	6.69	0.81	0.44
query29	12.54	3.95	3.26
query30	0.25	0.10	0.06
query31	2.83	0.61	0.38
query32	3.23	0.54	0.46
query33	3.00	3.02	3.05
query34	15.81	5.11	4.54
query35	4.57	4.55	4.60
query36	0.64	0.49	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.02	0.02	0.02
query40	0.17	0.14	0.12
query41	0.08	0.02	0.03
query42	0.03	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 102.86 s
Total hot run time: 30.28 s

if stmtId larger than Interger.MAX_VALUE, there is chance to return `Not supported such prepared statement` to the client which indicates the stmtId not exist.In order to fix this bug: 1. change id from original `stmtId` to `preparedStmtId` in ConnectContext, id which increment in session level and only increment when comming new prepared statement 2. rolling the id to fit the range [-2147483648, 2147483647)

…48116) Cherry-picked from #47950 Co-authored-by: lihangyu <lihangyu@selectdb.com>

if stmtId larger than Interger.MAX_VALUE, there is chance to return `Not supported such prepared statement` to the client which indicates the stmtId not exist.In order to fix this bug: 1. change id from original `stmtId` to `preparedStmtId` in ConnectContext, id which increment in session level and only increment when comming new prepared statement 2. rolling the id to fit the range [-2147483648, 2147483647)

cherry-pick from (#47950)

…3763) (apache#3775)

if stmtId larger than Interger.MAX_VALUE, there is chance to return `Not supported such prepared statement` to the client which indicates the stmtId not exist.In order to fix this bug: 1. change id from original `stmtId` to `preparedStmtId` in ConnectContext, id which increment in session level and only increment when comming new prepared statement 2. rolling the id to fit the range [-2147483648, 2147483647)

eldenmoon added dev/2.1.x dev/3.0.x labels Feb 16, 2025

wm1581066 added the usercase Important user case type label label Feb 17, 2025

add test
0560960

qidaye approved these changes Feb 17, 2025
View reviewed changes

github-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Feb 17, 2025

morrySnow approved these changes Feb 20, 2025
View reviewed changes

eldenmoon merged commit d422116 into apache:masterFeb 20, 2025

github-actionsBot mentioned this pull request Feb 20, 2025
branch-3.0: [BugFix](PreparedStatement) fix stmtId overflow #47950 #48116
Merged

github-actionsBot added the dev/2.1.x-conflict label Feb 20, 2025

dataroaring pushed a commit that referenced this pull request Feb 22, 2025
branch-3.0: [BugFix](PreparedStatement) fix stmtId overflow#47950(#…
3f15ddb
…48116) Cherry-picked from #47950 Co-authored-by: lihangyu <lihangyu@selectdb.com>

dataroaring added dev/3.0.5-merged and removed dev/3.0.x labels Feb 22, 2025

eldenmoon mentioned this pull request Feb 25, 2025
[BugFix](PreparedStatement) fix stmtId overflow (#47950) #48271
Merged

yiguolei pushed a commit that referenced this pull request Feb 25, 2025
[BugFix](PreparedStatement) fix stmtId overflow (#47950) (#48271)
24702bf
cherry-pick from (#47950)

yiguolei added dev/2.1.9-merged and removed dev/2.1.x dev/2.1.x-conflict labels Feb 25, 2025

hubgeter pushed a commit to hubgeter/doris that referenced this pull request Mar 12, 2025
[BugFix](PreparedStatement) fix stmtId overflowapache#47950(apache#…
4a3c19c
…3763) (apache#3775)

yiguolei mentioned this pull request Mar 25, 2025
2.1.9 Release Note #49443
Closed

Navigation Menu

Uh oh!

[BugFix](PreparedStatement) fix stmtId overflow - #47950

[BugFix](PreparedStatement) fix stmtId overflow#47950
eldenmoon merged 2 commits into
apache:masterfrom
eldenmoon:fix-prepared-id-overflow

eldenmoon commented Feb 16, 2025•
edited
Loading

Uh oh!

eldenmoon commented Feb 16, 2025

Uh oh!

Thearas commented Feb 16, 2025

Uh oh!

doris-robot commented Feb 16, 2025

Uh oh!

doris-robot commented Feb 16, 2025

Uh oh!

doris-robot commented Feb 16, 2025

Uh oh!

eldenmoon commented Feb 17, 2025

Uh oh!

qidaye left a comment

Uh oh!

github-actionsBot commented Feb 17, 2025

Uh oh!

github-actionsBot commented Feb 17, 2025

Uh oh!

doris-robot commented Feb 17, 2025

Uh oh!

doris-robot commented Feb 17, 2025

Uh oh!

doris-robot commented Feb 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Uh oh!

Conversation

eldenmoon commented Feb 16, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

eldenmoon commented Feb 16, 2025

Uh oh!

Thearas commented Feb 16, 2025

Uh oh!

doris-robot commented Feb 16, 2025

Uh oh!

doris-robot commented Feb 16, 2025

Uh oh!

doris-robot commented Feb 16, 2025

Uh oh!

eldenmoon commented Feb 17, 2025

Uh oh!

qidaye left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Feb 17, 2025

Uh oh!

github-actionsBot commented Feb 17, 2025

Uh oh!

doris-robot commented Feb 17, 2025

Uh oh!

doris-robot commented Feb 17, 2025

Uh oh!

doris-robot commented Feb 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

eldenmoon commented Feb 16, 2025•
edited
Loading