[fix](memory) Fix Allocator release memory to correct MemTracker after TLS attach task ends by xinyiZzz · Pull Request #39908 · apache/doris

xinyiZzz · 2024-08-26T04:23:06Z

Allocator save TLS MemTracker during first alloc, which is used to release memory after TLS attach task ends.

23:00:15 F20240824 22:56:49.773799 66432 thread_context.h:238] Check failed: doris::k_doris_exit || !doris::config::enable_memory_orphan_check || thread_mem_tracker()->label() != "Orphan" If you crash here, it means that SCOPED_ATTACH_TASK and SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER are not used correctly. starting position of each thread is expected to use SCOPED_ATTACH_TASK to bind a MemTrackerLimiter belonging to Query/Load/Compaction/Other Tasks, otherwise memory alloc using Doris Allocator in the thread will crash. If you want to switch MemTrackerLimiter during thread execution, please use SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER, do not repeat Attach. Of course, you can modify enable_memory_orphan_check=false in be.conf to avoid this crash.
23:00:15 *** Check failure stack trace: ***
23:00:15 @ 0x55645d3388e6 google::LogMessageFatal::~LogMessageFatal()
23:00:15 @ 0x5564439637c2 doris::ThreadContext::consume_memory()
23:00:15 @ 0x5564439914fe Allocator<>::release_memory()
23:00:15 @ 0x5564354be11e std::_Sp_counted_ptr<>::_M_dispose()
23:00:15 @ 0x55643557bc3b std::deque<>::pop_front()
23:00:15 @ 0x5564355756b1 doris::io::StreamLoadPipe::~StreamLoadPipe()
23:00:15 @ 0x5564354bfa77 doris::StreamLoadContext::~StreamLoadContext()
23:00:15 @ 0x556436ee5114 doris::HttpRequest::~HttpRequest()

doris-robot · 2024-08-26T04:23:10Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

xinyiZzz · 2024-08-26T04:23:12Z

run buildall

github-actions · 2024-08-26T04:30:43Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-08-26T05:46:16Z

TPC-H: Total hot run time: 38256 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 99c031a1bd24a847a8837b48152c72ab7f1109a9, data reload: false
------ Round 1 ----------------------------------
q1	18003	4674	4434	4434
q2	2615	178	172	172
q3	11769	1100	1185	1100
q4	11258	816	764	764
q5	8839	2923	2854	2854
q6	227	135	136	135
q7	973	612	583	583
q8	9359	2090	2093	2090
q9	7184	6570	6589	6570
q10	7008	2169	2241	2169
q11	474	242	260	242
q12	392	217	219	217
q13	17788	3033	3045	3033
q14	284	245	231	231
q15	541	480	496	480
q16	485	395	396	395
q17	1010	671	742	671
q18	7391	6799	6926	6799
q19	1388	1089	1006	1006
q20	688	331	329	329
q21	3882	3036	2989	2989
q22	1119	993	1013	993
Total cold run time: 112677 ms
Total hot run time: 38256 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4421	4244	4273	4244
q2	375	285	288	285
q3	2899	2719	2671	2671
q4	1922	1645	1656	1645
q5	5407	5404	5418	5404
q6	227	131	130	130
q7	2133	1727	1747	1727
q8	3221	3400	3330	3330
q9	8551	8511	8394	8394
q10	3466	3179	3207	3179
q11	593	475	475	475
q12	805	623	626	623
q13	11883	3014	3005	3005
q14	312	282	273	273
q15	522	491	479	479
q16	475	431	427	427
q17	1783	1507	1479	1479
q18	7752	7500	7410	7410
q19	1681	1494	1602	1494
q20	2074	1804	1819	1804
q21	5533	5288	5178	5178
q22	1093	1023	1032	1023
Total cold run time: 67128 ms
Total hot run time: 54679 ms

doris-robot · 2024-08-26T05:57:52Z

TPC-DS: Total hot run time: 186640 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 99c031a1bd24a847a8837b48152c72ab7f1109a9, data reload: false
query1	918	380	365	365
query2	6458	1996	1860	1860
query3	6652	217	219	217
query4	29440	23176	23080	23080
query5	4139	506	497	497
query6	271	167	176	167
query7	4585	300	290	290
query8	242	199	209	199
query9	8415	2483	2470	2470
query10	438	265	263	263
query11	17735	14974	15143	14974
query12	152	101	96	96
query13	1627	368	368	368
query14	8699	7240	7066	7066
query15	281	168	172	168
query16	8071	466	469	466
query17	1591	583	563	563
query18	2117	299	293	293
query19	227	153	152	152
query20	123	112	113	112
query21	212	112	104	104
query22	4439	4140	4050	4050
query23	34163	33378	33365	33365
query24	11177	2929	2894	2894
query25	654	418	399	399
query26	1174	161	164	161
query27	2500	287	280	280
query28	7333	2043	2048	2043
query29	836	435	426	426
query30	305	159	153	153
query31	985	787	763	763
query32	98	57	60	57
query33	763	298	336	298
query34	939	480	477	477
query35	888	723	731	723
query36	1110	907	935	907
query37	166	85	80	80
query38	4060	3879	3914	3879
query39	1451	1390	1391	1390
query40	201	118	115	115
query41	47	50	49	49
query42	117	99	102	99
query43	502	479	459	459
query44	1216	739	754	739
query45	203	165	167	165
query46	1105	759	723	723
query47	1842	1844	1817	1817
query48	378	289	290	289
query49	1097	429	427	427
query50	808	414	426	414
query51	7394	7024	7016	7016
query52	101	89	88	88
query53	257	182	176	176
query54	914	461	462	461
query55	77	88	77	77
query56	287	265	263	263
query57	1181	1081	1038	1038
query58	231	215	241	215
query59	3021	2861	2597	2597
query60	291	271	258	258
query61	99	96	103	96
query62	850	668	634	634
query63	226	182	178	178
query64	5221	2294	1743	1743
query65	3216	3177	3180	3177
query66	843	335	336	335
query67	15503	15239	15326	15239
query68	4462	564	551	551
query69	408	281	315	281
query70	1110	1146	1069	1069
query71	344	276	276	276
query72	6490	2312	2081	2081
query73	745	321	348	321
query74	9017	8799	8885	8799
query75	3412	2687	2713	2687
query76	2279	958	990	958
query77	511	315	323	315
query78	9604	9023	8991	8991
query79	1016	534	538	534
query80	710	507	499	499
query81	498	227	227	227
query82	232	138	143	138
query83	175	155	152	152
query84	224	83	79	79
query85	892	301	281	281
query86	322	282	307	282
query87	4470	4331	4312	4312
query88	3041	2353	2328	2328
query89	381	299	292	292
query90	1916	203	203	203
query91	126	102	100	100
query92	67	54	53	53
query93	1035	533	539	533
query94	933	292	292	292
query95	363	266	259	259
query96	590	269	277	269
query97	3211	3024	3148	3024
query98	220	200	196	196
query99	1535	1256	1271	1256
Total cold run time: 283001 ms
Total hot run time: 186640 ms

doris-robot · 2024-08-26T06:03:21Z

ClickBench: Total hot run time: 31.25 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 99c031a1bd24a847a8837b48152c72ab7f1109a9, data reload: false
query1	0.05	0.04	0.04
query2	0.09	0.05	0.04
query3	0.22	0.05	0.05
query4	1.69	0.08	0.08
query5	0.51	0.51	0.50
query6	1.15	0.73	0.73
query7	0.02	0.02	0.02
query8	0.06	0.04	0.04
query9	0.55	0.48	0.48
query10	0.54	0.53	0.53
query11	0.17	0.12	0.11
query12	0.15	0.12	0.12
query13	0.61	0.59	0.59
query14	0.76	0.79	0.77
query15	0.88	0.81	0.83
query16	0.37	0.35	0.38
query17	1.05	1.03	1.03
query18	0.21	0.20	0.20
query19	1.85	1.86	1.79
query20	0.01	0.01	0.01
query21	15.39	0.67	0.66
query22	4.13	6.97	2.37
query23	18.62	1.38	1.30
query24	2.05	0.23	0.22
query25	0.16	0.08	0.08
query26	0.26	0.18	0.18
query27	0.08	0.08	0.07
query28	13.20	1.03	1.00
query29	12.61	3.31	3.33
query30	0.25	0.06	0.06
query31	2.87	0.40	0.41
query32	3.25	0.48	0.48
query33	2.96	3.03	2.98
query34	17.01	4.37	4.40
query35	4.44	4.38	4.48
query36	0.66	0.49	0.50
query37	0.19	0.15	0.16
query38	0.16	0.15	0.15
query39	0.05	0.05	0.04
query40	0.16	0.13	0.13
query41	0.10	0.06	0.05
query42	0.06	0.06	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.65 s
Total hot run time: 31.25 s

xinyiZzz · 2024-08-26T09:52:44Z

run buildall

github-actions · 2024-08-26T09:56:43Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-08-26T09:59:54Z

clang-tidy review says "All clean, LGTM! 👍"

xinyiZzz · 2024-08-26T11:36:39Z

run buildall

github-actions · 2024-08-26T11:43:29Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-08-26T11:59:31Z

TPC-H: Total hot run time: 38447 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4efd07bb22835016e19cdc6b800edfe16d51db90, data reload: false
------ Round 1 ----------------------------------
q1	18765	4531	4385	4385
q2	2556	195	180	180
q3	11389	1176	1135	1135
q4	10691	674	662	662
q5	7756	2901	2919	2901
q6	232	141	144	141
q7	973	621	606	606
q8	9334	2054	2069	2054
q9	7295	6526	6554	6526
q10	6994	2237	2162	2162
q11	449	251	246	246
q12	399	227	231	227
q13	18520	3058	3032	3032
q14	288	239	236	236
q15	532	483	501	483
q16	534	448	458	448
q17	984	705	724	705
q18	7449	6795	7008	6795
q19	1405	1007	1067	1007
q20	690	346	341	341
q21	4788	3166	3271	3166
q22	1128	1009	1045	1009
Total cold run time: 113151 ms
Total hot run time: 38447 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4398	4358	4275	4275
q2	401	286	275	275
q3	2942	2600	2605	2600
q4	1931	1675	1617	1617
q5	5394	5421	5396	5396
q6	223	134	138	134
q7	2111	1753	1713	1713
q8	3203	3333	3331	3331
q9	8501	8476	8441	8441
q10	3470	3209	3164	3164
q11	596	502	507	502
q12	785	604	593	593
q13	11453	3062	3088	3062
q14	306	267	276	267
q15	538	474	486	474
q16	577	527	522	522
q17	1803	1496	1485	1485
q18	7644	7448	7385	7385
q19	1678	1613	1627	1613
q20	2086	1818	1822	1818
q21	5509	5350	5303	5303
q22	1132	1055	1029	1029
Total cold run time: 66681 ms
Total hot run time: 54999 ms

doris-robot · 2024-08-26T12:11:12Z

TPC-DS: Total hot run time: 186686 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4efd07bb22835016e19cdc6b800edfe16d51db90, data reload: false
query1	907	372	369	369
query2	6473	2002	1900	1900
query3	6650	207	216	207
query4	34260	23165	23174	23165
query5	4161	519	500	500
query6	247	161	160	160
query7	4581	309	293	293
query8	261	218	221	218
query9	8570	2495	2502	2495
query10	438	293	260	260
query11	17806	15123	15110	15110
query12	149	100	98	98
query13	1622	380	372	372
query14	9858	6776	7127	6776
query15	264	179	178	178
query16	8013	488	446	446
query17	1593	580	570	570
query18	2017	305	315	305
query19	264	139	143	139
query20	115	109	111	109
query21	211	109	104	104
query22	4375	4098	3967	3967
query23	33893	33400	33593	33400
query24	11108	2841	2910	2841
query25	624	386	385	385
query26	1224	161	157	157
query27	2542	275	276	275
query28	7454	2167	2140	2140
query29	809	420	402	402
query30	312	153	151	151
query31	1048	785	777	777
query32	100	56	57	56
query33	762	300	288	288
query34	974	485	484	484
query35	878	724	728	724
query36	1092	942	953	942
query37	164	102	97	97
query38	3954	3900	3803	3803
query39	1465	1361	1372	1361
query40	200	115	116	115
query41	49	49	44	44
query42	113	100	97	97
query43	501	455	460	455
query44	1251	742	760	742
query45	203	169	168	168
query46	1109	748	718	718
query47	1911	1843	1829	1829
query48	391	296	309	296
query49	1107	428	418	418
query50	810	416	410	410
query51	7224	7124	7021	7021
query52	97	89	120	89
query53	255	177	177	177
query54	940	447	447	447
query55	74	77	79	77
query56	277	257	251	251
query57	1181	1080	1054	1054
query58	235	213	236	213
query59	3008	2795	2799	2795
query60	303	270	262	262
query61	99	96	98	96
query62	828	656	667	656
query63	215	193	185	185
query64	5342	2310	1763	1763
query65	3208	3133	3148	3133
query66	1368	345	339	339
query67	15630	15324	15298	15298
query68	3563	565	566	565
query69	399	281	285	281
query70	1100	1056	1038	1038
query71	332	272	285	272
query72	6403	2307	2036	2036
query73	758	320	325	320
query74	9078	8879	8779	8779
query75	3439	2731	2645	2645
query76	1873	1011	1013	1011
query77	485	319	325	319
query78	9668	8942	8991	8942
query79	1024	539	537	537
query80	691	508	501	501
query81	471	241	233	233
query82	242	152	149	149
query83	194	154	148	148
query84	228	78	73	73
query85	683	278	278	278
query86	303	288	293	288
query87	4370	4321	4201	4201
query88	2887	2322	2311	2311
query89	372	289	291	289
query90	1852	199	197	197
query91	131	112	112	112
query92	65	56	55	55
query93	1036	550	540	540
query94	862	306	302	302
query95	366	348	260	260
query96	582	270	275	270
query97	3173	3120	3086	3086
query98	219	204	195	195
query99	1466	1317	1281	1281
Total cold run time: 287482 ms
Total hot run time: 186686 ms

doris-robot · 2024-08-26T12:16:41Z

ClickBench: Total hot run time: 31.84 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4efd07bb22835016e19cdc6b800edfe16d51db90, data reload: false
query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.06	0.05
query4	1.67	0.08	0.07
query5	0.52	0.52	0.51
query6	1.12	0.73	0.73
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.55	0.49	0.49
query10	0.54	0.54	0.54
query11	0.16	0.11	0.11
query12	0.15	0.13	0.12
query13	0.62	0.59	0.58
query14	1.69	1.74	1.68
query15	0.86	0.81	0.81
query16	0.38	0.37	0.37
query17	1.08	1.08	1.05
query18	0.21	0.20	0.20
query19	1.83	1.81	1.85
query20	0.01	0.01	0.01
query21	15.43	0.67	0.66
query22	3.91	6.74	1.96
query23	18.29	1.32	1.26
query24	2.14	0.23	0.23
query25	0.15	0.08	0.08
query26	0.29	0.18	0.18
query27	0.07	0.08	0.07
query28	13.21	1.02	1.01
query29	12.68	3.35	3.29
query30	0.25	0.05	0.06
query31	2.88	0.40	0.40
query32	3.24	0.49	0.49
query33	2.98	2.99	3.01
query34	17.09	4.42	4.39
query35	4.44	4.45	4.47
query36	0.66	0.47	0.46
query37	0.20	0.15	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.15	0.13	0.14
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.28 s
Total hot run time: 31.84 s

github-actions · 2024-08-26T12:54:07Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-08-26T12:54:08Z

PR approved by anyone and no changes requested.

xinyiZzz · 2024-08-26T13:39:24Z

DCHECK(doris::thread_context()->thread_mem_tracker()->type() != doris::MemTrackerLimiter::Type::QUERY || doris::thread_context()->thread_mem_tracker()->label() == mem_tracker_->label())
<< ", thread mem tracker label: "
<< doris::thread_context()->thread_mem_tracker()->label()
<< ", allocator tracker label: " << mem_tracker_->label();

F20240826 21:01:08.522873 135505 allocator.cpp:220] Check failed: doris::thread_context()->thread_mem_tracker()->label() == mem_tracker->label() , thread mem tracker label: Load#Id=6834273cfabe49cb-9d783fde25a0d61a, allocator tracker label: 8#0
21:11:13 *** Check failure stack trace: ***
21:11:13 @ 0x55a5a4d2a0d6 google::LogMessageFatal::~LogMessageFatal()
21:11:13 @ 0x55a5891a3a7d Allocator<>::consume_memory()
21:11:13 @ 0x55a5891a6213 Allocator<>::alloc_impl()
21:11:13 @ 0x55a5a72df17f orc::DataBuffer<>::reserve()
21:11:13 @ 0x55a5a72de421 orc::DataBuffer<>::resize()
21:11:13 @ 0x55a5a72def7f orc::DataBuffer<>::DataBuffer()
21:11:13 @ 0x55a5a72f3723 std::make_unique<>()
21:11:13 @ 0x55a5a72edfcf orc::createReader()
21:11:13 @ 0x55a58969edcb doris::vectorized::OrcReader::_create_file_reader()
21:11:13 @ 0x55a5896a0947 doris::vectorized::OrcReader::init_reader()
21:11:13 @ 0x55a5a300b14c doris::vectorized::VFileScanner::_get_next_reader()
21:11:13 @ 0x55a5a3001e92 doris::vectorized::VFileScanner::_get_block_wrapped()
21:11:13 @ 0x55a5a3000f5a doris::vectorized::VFileScanner::_get_block_impl()
21:11:13 @ 0x55a58a296eb9 doris::vectorized::VScanner::get_block()
21:11:13 @ 0x55a58a2962f5 doris::vectorized::VScanner::get_block_after_projects()
21:11:13 @ 0x55a58a248798 doris::vectorized::ScannerScheduler::_scanner_scan()
21:11:13 @ 0x55a58a24c1e0 _ZNSt17_Function_handlerIFvvEZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS2_14ScannerContextEES4_INS2_8ScanTaskEEENK3$_1clEvEUlvE_E9_M_invokeERKSt9_Any_data
21:11:13 @ 0x55a57316801e doris::ThreadPool::dispatch_thread()
21:11:13 @ 0x55a57314125f doris::Thread::supervise_thread()
21:11:13 @ 0x7f8dc7b17609 start_thread
21:11:13 @ 0x7f8dc7dc4133 clone
21:11:13 @ (nil) (unknown)
21:11:13 *** Query id: 6834273cfabe49cb-9d783fde25a0d61a ***
21:11:13 *** is nereids: 1 ***
21:11:13 *** tablet id: 0 ***
21:11:13 *** Aborted at 1724677269 (unix time) try "date -d @1724677269" if you are using GNU date ***
21:11:13 *** Current BE git commitID: fbd4a0087c ***
21:11:13 *** SIGABRT unknown detail explain (@0x209f7) received by PID 133623 (TID 135505 OR 0x7f8627a04700) from PID 133623; stack trace: ***
21:11:13 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/doris/be/src/common/signal_handler.h:421
21:11:13 1# 0x00007F8DC7CE8090 in /lib/x86_64-linux-gnu/libc.so.6
21:11:13 2# raise at ../sysdeps/unix/sysv/linux/raise.c:51
21:11:13 3# abort at /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:81
21:11:13 4# 0x000055A5A4D3104D in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be
21:11:13 5# google::LogMessage::SendToLog() in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be
21:11:13 6# google::LogMessage::Flush() in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be
21:11:13 7# google::LogMessageFatal::~LogMessageFatal() in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be
21:11:13 8# Allocator<false, false, false, RecordSizeMemoryAllocator>::consume_memory(unsigned long) at /root/doris/be/src/vec/common/allocator.cpp:220
21:11:13 9# Allocator<false, false, false, RecordSizeMemoryAllocator>::alloc_impl(unsigned long, unsigned long) at /root/doris/be/src/vec/common/allocator.h:251
21:11:13 10# orc::DataBuffer<char>::reserve(unsigned long) in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be
21:11:13 11# orc::DataBuffer<char>::resize(unsigned long) in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be
21:11:13 12# orc::DataBuffer<char>::DataBuffer(orc::MemoryPool&, unsigned long) in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be
21:11:13 13# std::_MakeUniq<orc::DataBuffer<char> >::__single_object std::make_unique<orc::DataBuffer<char>, orc::MemoryPool&, unsigned long&>(orc::MemoryPool&, unsigned long&) in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be
21:11:13 14# orc::createReader(std::unique_ptr<orc::InputStream, std::default_delete<orc::InputStream> >, orc::ReaderOptions const&) in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be
21:11:13 15# doris::vectorized::OrcReader::_create_file_reader() in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be
21:11:13 16# doris::vectorized::OrcReader::init_reader(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const*, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::variant<doris::ColumnValueRange<(doris::PrimitiveType)3>, doris::ColumnValueRange<(doris::PrimitiveType)4>, doris::ColumnValueRange<(doris::PrimitiveType)5>, doris::ColumnValueRange<(doris::PrimitiveType)6>, doris::ColumnValueRange<(doris::PrimitiveType)7>, doris::ColumnValueRange<(doris::PrimitiveType)36>, doris::ColumnValueRange<(doris::PrimitiveType)37>, doris::ColumnValueRange<(doris::PrimitiveType)15>, doris::ColumnValueRange<(doris::PrimitiveType)10>, doris::ColumnValueRange<(doris::PrimitiveType)23>, doris::ColumnValueRange<(doris::PrimitiveType)11>, doris::ColumnValueRange<(doris::PrimitiveType)25>, doris::ColumnValueRange<(doris::PrimitiveType)12>, doris::ColumnValueRange<(doris::PrimitiveType)26>, doris::ColumnValueRange<(doris::PrimitiveType)20>, doris::ColumnValueRange<(doris::PrimitiveType)2>, doris::ColumnValueRange<(doris::PrimitiveType)19>, doris::ColumnValueRange<(doris::PrimitiveType)28>, doris::ColumnValueRange<(doris::PrimitiveType)29>, doris::ColumnValueRange<(doris::PrimitiveType)30>, doris::ColumnValueRange<(doris::PrimitiveType)35> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::variant<doris::ColumnValueRange<(doris::PrimitiveType)3>, doris::ColumnValueRange<(doris::PrimitiveType)4>, doris::ColumnValueRange<(doris::PrimitiveType)5>, doris::ColumnValueRange<(doris::PrimitiveType)6>, doris::ColumnValueRange<(doris::PrimitiveType)7>, doris::ColumnValueRange<(doris::PrimitiveType)36>, doris::ColumnValueRange<(doris::PrimitiveType)37>, doris::ColumnValueRange<(doris::PrimitiveType)15>, doris::ColumnValueRange<(doris::PrimitiveType)10>, doris::ColumnValueRange<(doris::PrimitiveType)23>, doris::ColumnValueRange<(doris::PrimitiveType)11>, doris::ColumnValueRange<(doris::PrimitiveType)25>, doris::ColumnValueRange<(doris::PrimitiveType)12>, doris::ColumnValueRange<(doris::PrimitiveType)26>, doris::ColumnValueRange<(doris::PrimitiveType)20>, doris::ColumnValueRange<(doris::PrimitiveType)2>, doris::ColumnValueRange<(doris::PrimitiveType)19>, doris::ColumnValueRange<(doris::PrimitiveType)28>, doris::ColumnValueRange<(doris::PrimitiveType)29>, doris::ColumnValueRange<(doris::PrimitiveType)30>, doris::ColumnValueRange<(doris::PrimitiveType)35> > > > >*, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext> > > const&, bool, doris::TupleDescriptor const*, doris::RowDescriptor const*, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext> > > const*, std::unordered_map<int, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext> > >, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext> > > > > > const*, bool) at /root/doris/be/src/vec/exec/format/orc/vorc_reader.cpp:300
21:11:13 17# doris::vectorized::VFileScanner::_get_next_reader() at /root/doris/be/src/vec/exec/scan/vfile_scanner.cpp:906
21:11:13 18# doris::vectorized::VFileScanner::_get_block_wrapped(doris::RuntimeState*, doris::vectorized::Block*, bool*) in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be
21:11:13 19# doris::vectorized::VFileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /root/doris/be/src/vec/exec/scan/vfile_scanner.cpp:253
21:11:13 20# doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) in /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be
21:11:13 21# doris::vectorized::VScanner::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /root/doris/be/src/vec/exec/scan/vscanner.cpp:81
21:11:13 22# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) at /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:236
21:11:13 23# std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
21:11:13 24# doris::ThreadPool::dispatch_thread() at /root/doris/be/src/util/threadpool.cpp:551
21:11:13 25# doris::Thread::supervise_thread(void*) at /root/doris/be/src/util/thread.cpp:499
21:11:13 26# start_thread at /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:478
21:11:13 27# __clone at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

xinyiZzz · 2024-08-26T13:41:42Z

run buildall

xinyiZzz · 2024-08-27T11:30:43Z

run buildall

github-actions · 2024-08-27T11:37:58Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-08-27T13:19:43Z

TPC-H: Total hot run time: 37925 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c1541cb3a0ff8ccbee9631f20e61d6e600df3e7c, data reload: false
------ Round 1 ----------------------------------
q1	18243	4712	4395	4395
q2	2024	181	174	174
q3	11633	974	1077	974
q4	10296	758	724	724
q5	7787	2869	2767	2767
q6	233	137	136	136
q7	971	606	600	600
q8	9312	2082	2071	2071
q9	7270	6550	6591	6550
q10	7039	2196	2245	2196
q11	512	240	235	235
q12	394	224	220	220
q13	17769	3011	3050	3011
q14	286	231	230	230
q15	528	487	486	486
q16	607	499	490	490
q17	993	712	667	667
q18	7402	6796	6891	6796
q19	1396	1064	1065	1064
q20	716	346	327	327
q21	3956	2869	2804	2804
q22	1125	1017	1008	1008
Total cold run time: 110492 ms
Total hot run time: 37925 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4387	4331	4301	4301
q2	381	289	278	278
q3	2855	2712	2700	2700
q4	1963	1702	1653	1653
q5	5653	5650	5683	5650
q6	230	135	140	135
q7	2228	1824	1848	1824
q8	3323	3390	3425	3390
q9	8914	8850	8851	8850
q10	3566	3427	3372	3372
q11	607	508	512	508
q12	809	643	647	643
q13	14299	3126	3183	3126
q14	325	299	285	285
q15	545	501	494	494
q16	638	563	574	563
q17	1854	1558	1542	1542
q18	8124	7767	7845	7767
q19	1752	1695	1587	1587
q20	2180	1911	1890	1890
q21	5852	5564	5593	5564
q22	1171	1061	1027	1027
Total cold run time: 71656 ms
Total hot run time: 57149 ms

doris-robot · 2024-08-27T13:31:54Z

TPC-DS: Total hot run time: 192644 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c1541cb3a0ff8ccbee9631f20e61d6e600df3e7c, data reload: false
query1	1255	887	854	854
query2	6276	2060	2021	2021
query3	10707	4119	4031	4031
query4	60185	26461	23273	23273
query5	5373	501	509	501
query6	400	166	173	166
query7	5741	294	307	294
query8	279	201	206	201
query9	8498	2474	2484	2474
query10	470	284	252	252
query11	15643	15016	15039	15016
query12	156	101	105	101
query13	1505	396	389	389
query14	11154	6956	6523	6523
query15	230	193	178	178
query16	7566	472	513	472
query17	1135	595	614	595
query18	2036	306	306	306
query19	300	164	155	155
query20	117	118	113	113
query21	221	109	109	109
query22	4456	4405	4385	4385
query23	34214	33690	33366	33366
query24	5991	3001	2882	2882
query25	604	395	369	369
query26	680	158	152	152
query27	1780	280	281	280
query28	3801	2137	2116	2116
query29	680	402	407	402
query30	239	150	150	150
query31	929	769	782	769
query32	83	52	59	52
query33	460	286	291	286
query34	875	488	472	472
query35	865	713	724	713
query36	1048	905	920	905
query37	141	90	90	90
query38	3994	3875	3960	3875
query39	1429	1397	1391	1391
query40	197	117	116	116
query41	47	76	43	43
query42	111	98	98	98
query43	509	489	501	489
query44	1090	747	760	747
query45	198	165	171	165
query46	1113	777	745	745
query47	1880	1766	1808	1766
query48	383	298	310	298
query49	758	424	432	424
query50	834	402	401	401
query51	7156	7027	7019	7019
query52	101	87	87	87
query53	258	182	180	180
query54	563	455	452	452
query55	82	78	79	78
query56	290	253	251	251
query57	1182	1071	1080	1071
query58	229	231	224	224
query59	3138	2990	2779	2779
query60	295	314	283	283
query61	103	99	104	99
query62	749	662	656	656
query63	224	190	191	190
query64	2757	692	669	669
query65	3222	3122	3186	3122
query66	662	331	328	328
query67	15366	15451	15229	15229
query68	2910	576	585	576
query69	401	281	279	279
query70	1224	1083	1038	1038
query71	343	275	270	270
query72	6220	4075	4037	4037
query73	753	330	336	330
query74	9188	8899	8943	8899
query75	3340	2713	2697	2697
query76	1528	1023	962	962
query77	514	332	339	332
query78	9643	9884	9188	9188
query79	1076	544	541	541
query80	691	569	523	523
query81	468	235	234	234
query82	246	155	151	151
query83	176	153	150	150
query84	268	76	85	76
query85	710	293	286	286
query86	299	277	313	277
query87	4381	4339	4288	4288
query88	3096	2339	2313	2313
query89	382	287	294	287
query90	2047	191	193	191
query91	122	99	100	99
query92	62	53	53	53
query93	1049	532	536	532
query94	728	311	313	311
query95	354	261	254	254
query96	590	269	267	267
query97	3169	3052	3113	3052
query98	218	208	203	203
query99	1522	1267	1260	1260
Total cold run time: 302731 ms
Total hot run time: 192644 ms

doris-robot · 2024-08-27T13:37:30Z

ClickBench: Total hot run time: 32.28 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c1541cb3a0ff8ccbee9631f20e61d6e600df3e7c, data reload: false
query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.06
query4	1.68	0.08	0.07
query5	0.50	0.49	0.49
query6	1.14	0.72	0.74
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.48	0.48
query10	0.54	0.55	0.55
query11	0.16	0.12	0.12
query12	0.16	0.12	0.12
query13	0.61	0.58	0.59
query14	2.04	2.04	2.08
query15	0.84	0.82	0.81
query16	0.35	0.38	0.38
query17	1.01	1.07	1.07
query18	0.21	0.21	0.21
query19	1.96	1.80	1.76
query20	0.01	0.01	0.00
query21	15.39	0.67	0.66
query22	4.53	7.61	1.88
query23	18.25	1.42	1.33
query24	2.21	0.23	0.22
query25	0.16	0.08	0.09
query26	0.27	0.17	0.18
query27	0.07	0.08	0.09
query28	13.20	1.03	1.00
query29	12.62	3.44	3.43
query30	0.24	0.05	0.06
query31	2.87	0.41	0.40
query32	3.25	0.49	0.48
query33	2.98	2.97	3.00
query34	17.02	4.43	4.46
query35	4.50	4.50	4.45
query36	0.65	0.49	0.47
query37	0.20	0.16	0.16
query38	0.15	0.16	0.15
query39	0.05	0.04	0.03
query40	0.15	0.13	0.12
query41	0.10	0.05	0.05
query42	0.05	0.05	0.05
query43	0.05	0.05	0.04
Total cold run time: 111.13 s
Total hot run time: 32.28 s

…r TLS attach task ends (apache#39908) Allocator save TLS MemTracker during first alloc, which is used to release memory after TLS attach task ends. ``` 23:00:15 F20240824 22:56:49.773799 66432 thread_context.h:238] Check failed: doris::k_doris_exit || !doris::config::enable_memory_orphan_check || thread_mem_tracker()->label() != "Orphan" If you crash here, it means that SCOPED_ATTACH_TASK and SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER are not used correctly. starting position of each thread is expected to use SCOPED_ATTACH_TASK to bind a MemTrackerLimiter belonging to Query/Load/Compaction/Other Tasks, otherwise memory alloc using Doris Allocator in the thread will crash. If you want to switch MemTrackerLimiter during thread execution, please use SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER, do not repeat Attach. Of course, you can modify enable_memory_orphan_check=false in be.conf to avoid this crash. 23:00:15 *** Check failure stack trace: *** 23:00:15 @ 0x55645d3388e6 google::LogMessageFatal::~LogMessageFatal() 23:00:15 @ 0x5564439637c2 doris::ThreadContext::consume_memory() 23:00:15 @ 0x5564439914fe Allocator<>::release_memory() 23:00:15 @ 0x5564354be11e std::_Sp_counted_ptr<>::_M_dispose() 23:00:15 @ 0x55643557bc3b std::deque<>::pop_front() 23:00:15 @ 0x5564355756b1 doris::io::StreamLoadPipe::~StreamLoadPipe() 23:00:15 @ 0x5564354bfa77 doris::StreamLoadContext::~StreamLoadContext() 23:00:15 @ 0x556436ee5114 doris::HttpRequest::~HttpRequest() ```

…ns is equal to 0 (#40850) pick #38960 #39908 #40043 #40092 #40016 #40439 --------- Co-authored-by: hui lai <1353307710@qq.com> Co-authored-by: yiguolei <676222867@qq.com>

…r TLS attach task ends (#39908) Allocator save TLS MemTracker during first alloc, which is used to release memory after TLS attach task ends. ``` 23:00:15 F20240824 22:56:49.773799 66432 thread_context.h:238] Check failed: doris::k_doris_exit || !doris::config::enable_memory_orphan_check || thread_mem_tracker()->label() != "Orphan" If you crash here, it means that SCOPED_ATTACH_TASK and SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER are not used correctly. starting position of each thread is expected to use SCOPED_ATTACH_TASK to bind a MemTrackerLimiter belonging to Query/Load/Compaction/Other Tasks, otherwise memory alloc using Doris Allocator in the thread will crash. If you want to switch MemTrackerLimiter during thread execution, please use SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER, do not repeat Attach. Of course, you can modify enable_memory_orphan_check=false in be.conf to avoid this crash. 23:00:15 *** Check failure stack trace: *** 23:00:15 @ 0x55645d3388e6 google::LogMessageFatal::~LogMessageFatal() 23:00:15 @ 0x5564439637c2 doris::ThreadContext::consume_memory() 23:00:15 @ 0x5564439914fe Allocator<>::release_memory() 23:00:15 @ 0x5564354be11e std::_Sp_counted_ptr<>::_M_dispose() 23:00:15 @ 0x55643557bc3b std::deque<>::pop_front() 23:00:15 @ 0x5564355756b1 doris::io::StreamLoadPipe::~StreamLoadPipe() 23:00:15 @ 0x5564354bfa77 doris::StreamLoadContext::~StreamLoadContext() 23:00:15 @ 0x556436ee5114 doris::HttpRequest::~HttpRequest() ```

…r TLS attach task ends (apache#39908) Allocator save TLS MemTracker during first alloc, which is used to release memory after TLS attach task ends. ``` 23:00:15 F20240824 22:56:49.773799 66432 thread_context.h:238] Check failed: doris::k_doris_exit || !doris::config::enable_memory_orphan_check || thread_mem_tracker()->label() != "Orphan" If you crash here, it means that SCOPED_ATTACH_TASK and SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER are not used correctly. starting position of each thread is expected to use SCOPED_ATTACH_TASK to bind a MemTrackerLimiter belonging to Query/Load/Compaction/Other Tasks, otherwise memory alloc using Doris Allocator in the thread will crash. If you want to switch MemTrackerLimiter during thread execution, please use SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER, do not repeat Attach. Of course, you can modify enable_memory_orphan_check=false in be.conf to avoid this crash. 23:00:15 *** Check failure stack trace: *** 23:00:15 @ 0x55645d3388e6 google::LogMessageFatal::~LogMessageFatal() 23:00:15 @ 0x5564439637c2 doris::ThreadContext::consume_memory() 23:00:15 @ 0x5564439914fe Allocator<>::release_memory() 23:00:15 @ 0x5564354be11e std::_Sp_counted_ptr<>::_M_dispose() 23:00:15 @ 0x55643557bc3b std::deque<>::pop_front() 23:00:15 @ 0x5564355756b1 doris::io::StreamLoadPipe::~StreamLoadPipe() 23:00:15 @ 0x5564354bfa77 doris::StreamLoadContext::~StreamLoadContext() 23:00:15 @ 0x556436ee5114 doris::HttpRequest::~HttpRequest() ```

xinyiZzz requested review from dataroaring, gavinchou and platoneko as code owners August 26, 2024 04:23

yiguolei reviewed Aug 26, 2024
View reviewed changes

Comment threadbe/src/common/config.cpp

yiguolei reviewed Aug 26, 2024
View reviewed changes

Comment threadbe/src/common/config.h Outdated

xinyiZzz changed the title ~~[fix](bytebuffer) Fix ByteBuffer use Allocator~~[fix](memory) Fix Allocator release memory to correct MemTracker after TLS attach task endsAug 26, 2024

yiguolei reviewed Aug 26, 2024
View reviewed changes

Comment threadbe/src/vec/common/allocator.h Outdated

yiguolei reviewed Aug 26, 2024
View reviewed changes

Comment threadbe/src/vec/common/allocator.cpp Outdated

yiguolei previously approved these changes Aug 26, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Aug 26, 2024

github-actionsBot added the reviewed label Aug 26, 2024

xinyiZzz dismissed yiguolei’s stale review via d9baccdAugust 26, 2024 13:41

Merge branch 'master' into 20240824_fix_orphan
c1541cb

yiguolei approved these changes Aug 27, 2024
View reviewed changes

jacktengg approved these changes Aug 27, 2024
View reviewed changes

yiguolei approved these changes Aug 27, 2024
View reviewed changes

gavinchou approved these changes Aug 27, 2024
View reviewed changes

gavinchou merged commit c6b2cc7 into apache:masterAug 27, 2024

gavinchou added dev/2.0.x dev/3.0.x dev/2.1.x and removed dev/2.0.x labels Aug 28, 2024

xinyiZzz mentioned this pull request Sep 9, 2024
[branch-2.1](memory) Pick query memory tracker value is equal to 0 and exception handle #40564
Closed

xinyiZzz mentioned this pull request Sep 14, 2024
[branch-2.1](memory) When Load ends, check memory tracker value returns is equal to 0 #40850
Merged

xinyiZzz added dev/2.1.7-merged and removed dev/2.1.x labels Sep 20, 2024

dataroaring added dev/3.0.3-merged and removed dev/3.0.x labels Oct 9, 2024

Uh oh!

Conversation

xinyiZzz commented Aug 26, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

doris-robot commented Aug 26, 2024

Uh oh!

xinyiZzz commented Aug 26, 2024

Uh oh!

github-actionsBot commented Aug 26, 2024

Uh oh!

doris-robot commented Aug 26, 2024

Uh oh!

doris-robot commented Aug 26, 2024

Uh oh!

doris-robot commented Aug 26, 2024

Uh oh!

Uh oh!

Uh oh!

xinyiZzz commented Aug 26, 2024

Uh oh!

github-actionsBot commented Aug 26, 2024

Uh oh!

github-actionsBot commented Aug 26, 2024

Uh oh!

xinyiZzz commented Aug 26, 2024

Uh oh!

github-actionsBot commented Aug 26, 2024

Uh oh!

doris-robot commented Aug 26, 2024

Uh oh!

doris-robot commented Aug 26, 2024

Uh oh!

doris-robot commented Aug 26, 2024

Uh oh!

Uh oh!

Uh oh!

github-actionsBot commented Aug 26, 2024

Uh oh!

github-actionsBot commented Aug 26, 2024

Uh oh!

xinyiZzz commented Aug 26, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xinyiZzz commented Aug 26, 2024

Uh oh!

xinyiZzz commented Aug 27, 2024

Uh oh!

github-actionsBot commented Aug 27, 2024

Uh oh!

doris-robot commented Aug 27, 2024

Uh oh!

doris-robot commented Aug 27, 2024

Uh oh!

doris-robot commented Aug 27, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

xinyiZzz commented Aug 26, 2024•
edited
Loading

xinyiZzz commented Aug 26, 2024•
edited
Loading