[bug](udaf) fix memory leak in the java udaf by zhangstar333 · Pull Request #32630 · apache/doris

zhangstar333 · 2024-03-21T10:22:01Z

Proposed changes

before:
if someone call destroy function with place,
when place != _exec_place maybe do nothing ,
when place == _exec_place, do destroy of all place,
as we not want call JNI every times, so will destroy all place at once.
but not call this->data(place).~Data(); this maybe not release the use memory of std::string serialize_data.

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

doris-robot · 2024-03-21T10:22:07Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

zhangstar333 · 2024-03-21T10:22:21Z

run buildall

github-actions

clang-tidy made some suggestions

github-actions · 2024-03-21T10:27:21Z

- ~AggregateJavaUdafData() {
+ ~AggregateJavaUdafData() = default;
+
+ Status close_and_delete_object() {


warning: method 'close_and_delete_object' can be made static [readability-convert-member-functions-to-static]
Suggested change
Statusclose_and_delete_object() {
staticStatusclose_and_delete_object() {

doris-robot · 2024-03-21T10:49:03Z

TPC-H: Total hot run time: 37798 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8db8cce56bc4bc3ac050502cc16d52a5cf92653c, data reload: false
------ Round 1 ----------------------------------
q1	17641	4241	4129	4129
q2	2049	151	154	151
q3	10587	1094	1174	1094
q4	10231	798	826	798
q5	7516	3004	2959	2959
q6	206	124	122	122
q7	1047	571	561	561
q8	9551	2024	1988	1988
q9	8624	6480	6480	6480
q10	8408	3431	3490	3431
q11	420	236	219	219
q12	395	196	193	193
q13	17785	2877	2849	2849
q14	242	203	217	203
q15	491	457	463	457
q16	457	357	361	357
q17	973	586	592	586
q18	7251	6441	6398	6398
q19	1566	1409	1485	1409
q20	540	246	238	238
q21	3557	2911	2887	2887
q22	339	306	289	289
Total cold run time: 109876 ms
Total hot run time: 37798 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4091	4063	4041	4041
q2	317	227	221	221
q3	3002	2852	2881	2852
q4	1879	1556	1535	1535
q5	5256	5253	5253	5253
q6	193	114	114	114
q7	2300	1851	1824	1824
q8	3168	3314	3302	3302
q9	8623	8555	8639	8555
q10	3694	3726	3690	3690
q11	545	440	430	430
q12	715	547	535	535
q13	13734	2843	2839	2839
q14	275	248	249	248
q15	485	439	454	439
q16	453	415	414	414
q17	1744	1496	1468	1468
q18	7435	7121	7109	7109
q19	1599	1534	1517	1517
q20	1916	1723	1697	1697
q21	4874	4589	4624	4589
q22	548	442	456	442
Total cold run time: 66846 ms
Total hot run time: 53114 ms

doris-robot · 2024-03-21T10:52:06Z

TeamCity be ut coverage result:
Function Coverage: 35.27% (8730/24751)
Line Coverage: 27.09% (71495/263896)
Region Coverage: 26.32% (37082/140885)
Branch Coverage: 23.24% (18967/81626)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8db8cce56bc4bc3ac050502cc16d52a5cf92653c_8db8cce56bc4bc3ac050502cc16d52a5cf92653c/report/index.html

doris-robot · 2024-03-21T11:00:50Z

TPC-DS: Total hot run time: 180165 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8db8cce56bc4bc3ac050502cc16d52a5cf92653c, data reload: false
query1	916	368	335	335
query2	7319	2033	1925	1925
query3	6705	210	209	209
query4	31007	20615	20755	20615
query5	4347	427	500	427
query6	261	199	168	168
query7	4621	293	287	287
query8	225	164	179	164
query9	9943	2276	2248	2248
query10	459	239	246	239
query11	17063	14153	14198	14153
query12	135	88	85	85
query13	1631	410	414	410
query14	11624	10761	10094	10094
query15	274	196	192	192
query16	7686	250	252	250
query17	1910	544	521	521
query18	2083	278	271	271
query19	303	147	152	147
query20	92	90	82	82
query21	198	125	123	123
query22	4625	4470	4499	4470
query23	31800	31086	31053	31053
query24	10758	2799	2837	2799
query25	614	379	387	379
query26	1238	150	157	150
query27	2866	355	350	350
query28	7973	1911	1834	1834
query29	887	642	620	620
query30	303	146	148	146
query31	975	729	718	718
query32	95	61	56	56
query33	765	306	240	240
query34	983	457	461	457
query35	818	610	596	596
query36	995	828	891	828
query37	121	76	73	73
query38	3505	3438	3401	3401
query39	1428	1346	1379	1346
query40	205	111	107	107
query41	48	43	46	43
query42	100	96	91	91
query43	483	451	428	428
query44	1387	708	710	708
query45	267	256	256	256
query46	1089	681	682	681
query47	1678	1593	1586	1586
query48	440	349	347	347
query49	1120	324	322	322
query50	761	360	366	360
query51	6722	6550	6618	6550
query52	98	92	90	90
query53	340	267	268	267
query54	296	236	241	236
query55	80	75	76	75
query56	255	225	228	225
query57	1107	1012	997	997
query58	216	195	200	195
query59	2774	2423	2709	2423
query60	264	256	241	241
query61	99	93	95	93
query62	583	404	383	383
query63	288	270	269	269
query64	5660	3919	3889	3889
query65	3112	2966	3006	2966
query66	1444	368	347	347
query67	15082	14542	14453	14453
query68	6620	531	515	515
query69	600	370	375	370
query70	1239	1177	1140	1140
query71	478	277	287	277
query72	6538	2805	2640	2640
query73	728	322	316	316
query74	7076	6597	6635	6597
query75	3856	2768	2782	2768
query76	4181	927	966	927
query77	582	269	260	260
query78	10333	9696	9604	9604
query79	9381	517	520	517
query80	1532	412	417	412
query81	538	209	210	209
query82	1436	196	212	196
query83	226	138	137	137
query84	285	82	78	78
query85	1520	347	313	313
query86	477	296	307	296
query87	3824	3530	3520	3520
query88	4955	2286	2259	2259
query89	527	379	360	360
query90	1972	174	174	174
query91	168	131	132	131
query92	65	50	49	49
query93	7413	497	477	477
query94	1169	177	172	172
query95	423	338	319	319
query96	592	271	274	271
query97	3072	2870	2847	2847
query98	213	205	198	198
query99	1282	727	739	727
Total cold run time: 308850 ms
Total hot run time: 180165 ms

doris-robot · 2024-03-21T11:04:03Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 8db8cce56bc4bc3ac050502cc16d52a5cf92653c with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 20.9 seconds inserted 10000000 Rows, about 478K ops/s

zhangstar333 · 2024-03-22T03:58:48Z

run buildall

doris-robot · 2024-03-22T04:16:50Z

TPC-H: Total hot run time: 38070 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2cf8d102e263709d26436ef1276d6f8875784819, data reload: false
------ Round 1 ----------------------------------
q1	17605	4306	4126	4126
q2	2023	153	143	143
q3	10643	1229	1218	1218
q4	10776	826	836	826
q5	7461	2959	3276	2959
q6	208	127	122	122
q7	1042	590	576	576
q8	9346	2052	1980	1980
q9	7013	6491	6428	6428
q10	8368	3346	3511	3346
q11	425	221	213	213
q12	359	199	189	189
q13	17787	2852	2858	2852
q14	235	210	203	203
q15	509	464	452	452
q16	453	359	358	358
q17	964	646	589	589
q18	7186	6601	6582	6582
q19	2918	1416	1420	1416
q20	553	262	253	253
q21	3537	2942	2977	2942
q22	342	297	297	297
Total cold run time: 109753 ms
Total hot run time: 38070 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4106	4071	4124	4071
q2	319	229	228	228
q3	2964	2857	2837	2837
q4	1828	1556	1538	1538
q5	5253	5324	5248	5248
q6	193	116	115	115
q7	2274	1853	1862	1853
q8	3181	3311	3306	3306
q9	8595	8581	8605	8581
q10	3849	3682	3662	3662
q11	546	438	437	437
q12	725	538	542	538
q13	16922	2880	2869	2869
q14	272	264	257	257
q15	503	447	458	447
q16	467	410	422	410
q17	1735	1486	1485	1485
q18	7586	7297	7095	7095
q19	1680	1554	1546	1546
q20	1952	1733	1712	1712
q21	4960	4717	4514	4514
q22	531	478	433	433
Total cold run time: 70441 ms
Total hot run time: 53182 ms

doris-robot · 2024-03-22T04:18:36Z

TeamCity be ut coverage result:
Function Coverage: 35.27% (8733/24758)
Line Coverage: 27.08% (71524/264152)
Region Coverage: 26.32% (37111/140996)
Branch Coverage: 23.22% (18979/81738)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2cf8d102e263709d26436ef1276d6f8875784819_2cf8d102e263709d26436ef1276d6f8875784819/report/index.html

doris-robot · 2024-03-22T04:28:49Z

TPC-DS: Total hot run time: 181824 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2cf8d102e263709d26436ef1276d6f8875784819, data reload: false
query1	938	368	349	349
query2	7403	2016	1979	1979
query3	6710	217	214	214
query4	31262	20696	20773	20696
query5	4310	412	405	405
query6	278	192	176	176
query7	4655	292	286	286
query8	228	175	169	169
query9	9407	2345	2332	2332
query10	455	231	247	231
query11	14516	14212	14278	14212
query12	137	100	87	87
query13	1630	428	415	415
query14	13136	11189	11107	11107
query15	256	209	196	196
query16	8217	262	257	257
query17	1986	572	546	546
query18	2110	298	278	278
query19	344	156	154	154
query20	97	88	84	84
query21	201	135	132	132
query22	4634	4518	4441	4441
query23	32018	31099	30739	30739
query24	11640	2875	2792	2792
query25	663	380	381	380
query26	1842	152	155	152
query27	3166	355	356	355
query28	7838	1889	1915	1889
query29	1039	638	638	638
query30	307	149	147	147
query31	970	750	722	722
query32	95	59	58	58
query33	775	256	252	252
query34	1179	464	479	464
query35	828	602	604	602
query36	1000	881	895	881
query37	277	78	75	75
query38	3613	3405	3447	3405
query39	1406	1397	1374	1374
query40	289	117	111	111
query41	50	47	49	47
query42	105	96	96	96
query43	499	449	456	449
query44	1152	747	719	719
query45	270	277	266	266
query46	1104	671	689	671
query47	1665	1591	1614	1591
query48	462	354	355	354
query49	1234	335	343	335
query50	767	362	365	362
query51	6641	6600	6576	6576
query52	120	92	95	92
query53	344	276	279	276
query54	327	252	249	249
query55	85	81	78	78
query56	256	236	239	236
query57	1112	1021	1022	1021
query58	242	209	209	209
query59	2785	2643	2641	2641
query60	268	252	266	252
query61	132	121	122	121
query62	600	384	394	384
query63	304	276	276	276
query64	6402	3969	3859	3859
query65	3030	3007	2988	2988
query66	1453	345	330	330
query67	14792	14411	14300	14300
query68	9790	536	547	536
query69	680	381	377	377
query70	1366	1203	1186	1186
query71	547	281	272	272
query72	6709	2666	2510	2510
query73	1587	318	317	317
query74	7057	6616	6592	6592
query75	5550	2968	2849	2849
query76	5870	876	927	876
query77	779	269	255	255
query78	10291	9847	9720	9720
query79	11167	522	529	522
query80	1350	388	384	384
query81	504	211	209	209
query82	346	201	200	200
query83	206	146	144	144
query84	285	79	81	79
query85	1114	339	311	311
query86	348	310	285	285
query87	3710	3532	3538	3532
query88	5120	2319	2295	2295
query89	484	363	363	363
query90	2136	177	174	174
query91	181	135	135	135
query92	63	47	46	46
query93	6094	490	486	486
query94	1371	180	175	175
query95	425	332	333	332
query96	597	271	270	270
query97	3075	2885	2901	2885
query98	224	213	202	202
query99	962	759	745	745
Total cold run time: 318044 ms
Total hot run time: 181824 ms

doris-robot · 2024-03-22T04:32:02Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 2cf8d102e263709d26436ef1276d6f8875784819 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.8 seconds inserted 10000000 Rows, about 458K ops/s

HappenLee

LGTM

github-actions · 2024-03-22T10:05:48Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-03-22T10:05:50Z

PR approved by anyone and no changes requested.

fix memory leak in the java udaf

…2702)

fix memory leak in the java udaf

update
8db8cce

github-actionsBot reviewed Mar 21, 2024
View reviewed changes

HappenLee reviewed Mar 22, 2024
View reviewed changes

Comment threadbe/src/vec/aggregate_functions/aggregate_function_java_udaf.h Outdated

HappenLee added the dev/2.0.x label Mar 22, 2024

update comment
2cf8d10

xiaokang added the p0_b label Mar 22, 2024

HappenLee approved these changes Mar 22, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Mar 22, 2024

github-actionsBot added the reviewed label Mar 22, 2024

BiteTheDDDDt approved these changes Mar 22, 2024
View reviewed changes

BiteTheDDDDt merged commit 70919f5 into apache:masterMar 22, 2024

zhangstar333 added a commit to zhangstar333/incubator-doris that referenced this pull request Mar 22, 2024
[bug](udaf) fix memory leak in the java udaf (apache#32630)
9c08221
fix memory leak in the java udaf

zhangstar333 mentioned this pull request Mar 22, 2024
[cherry-pick](udaf) fix memory leak in the java udaf (#32630) #32702
Merged

xiaokang pushed a commit that referenced this pull request Mar 22, 2024
[bug](udaf) fix memory leak in the java udaf (#32630) (#32702)
7a52cbb

xiaokang added dev/2.0.7-merged and removed dev/2.0.x labels Mar 22, 2024

yiguolei pushed a commit that referenced this pull request Mar 24, 2024
[bug](udaf) fix memory leak in the java udaf (#32630)
a55b5ea
fix memory leak in the java udaf

yiguolei mentioned this pull request Mar 24, 2024
Release notes 2.1.1 #32736
Closed

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
[bug](udaf) fix memory leak in the java udaf (apache#32630) (apache#3…
287ffb6
…2702)

HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
[bug](udaf) fix memory leak in the java udaf (apache#32630)
4d3116e
fix memory leak in the java udaf

	Statusclose_and_delete_object() {
	staticStatusclose_and_delete_object() {

Uh oh!

Conversation

zhangstar333 commented Mar 21, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

doris-robot commented Mar 21, 2024

Uh oh!

zhangstar333 commented Mar 21, 2024

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBotMar 21, 2024

Choose a reason for hiding this comment

Uh oh!

doris-robot commented Mar 21, 2024

Uh oh!

doris-robot commented Mar 21, 2024

Uh oh!

doris-robot commented Mar 21, 2024

Uh oh!

doris-robot commented Mar 21, 2024

Uh oh!

Uh oh!

zhangstar333 commented Mar 22, 2024

Uh oh!

doris-robot commented Mar 22, 2024

Uh oh!

doris-robot commented Mar 22, 2024

Uh oh!

doris-robot commented Mar 22, 2024

Uh oh!

doris-robot commented Mar 22, 2024

Uh oh!

HappenLee left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Mar 22, 2024

Uh oh!

github-actionsBot commented Mar 22, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

zhangstar333 commented Mar 21, 2024•
edited
Loading