[enhancement](errmsg) Make stream load error message more clear and change some Errors stacktrace by zclllyybb · Pull Request #38978 · apache/doris

zclllyybb · 2024-08-06T13:15:47Z

Proposed changes

Issue Number: close #xxx

introduce ERROR_CTOR_NOSTACK to make some Doris' native error don't print stacktrace by default.
when streamload exceed limit, just tell user which conf to change

doris-robot · 2024-08-06T13:15:53Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

zclllyybb · 2024-08-06T13:20:00Z

run buildall

github-actions · 2024-08-06T13:22:33Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-08-06T13:27:11Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-08-06T13:44:00Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-08-06T13:44:02Z

PR approved by anyone and no changes requested.

doris-robot · 2024-08-06T14:54:52Z

TPC-H: Total hot run time: 41968 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 430be09f8d87842502c150362971b65ca30b1c42, data reload: false
------ Round 1 ----------------------------------
q1	17612	4135	4097	4097
q2	2040	202	213	202
q3	10510	1287	1361	1287
q4	10149	867	991	867
q5	7651	3017	3007	3007
q6	222	139	138	138
q7	1068	661	628	628
q8	9448	1947	1987	1947
q9	8520	6633	6625	6625
q10	8734	3855	3853	3853
q11	434	255	258	255
q12	406	236	230	230
q13	17767	2957	2961	2957
q14	277	242	237	237
q15	531	492	506	492
q16	522	400	389	389
q17	991	968	956	956
q18	7970	7413	7214	7214
q19	1593	1232	1220	1220
q20	574	320	341	320
q21	5367	4761	4813	4761
q22	356	287	286	286
Total cold run time: 112742 ms
Total hot run time: 41968 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4086	3993	4001	3993
q2	329	233	226	226
q3	3007	3020	3168	3020
q4	1997	2077	1984	1984
q5	5620	5510	5473	5473
q6	222	135	135	135
q7	2145	1785	1835	1785
q8	3341	3387	3388	3387
q9	8719	8761	8758	8758
q10	3942	3940	3957	3940
q11	556	462	478	462
q12	783	621	573	573
q13	16274	3131	3121	3121
q14	286	277	272	272
q15	548	499	498	498
q16	451	425	428	425
q17	1798	1751	1749	1749
q18	8168	7776	7715	7715
q19	1813	1732	1722	1722
q20	2071	1848	1840	1840
q21	5796	5586	5290	5290
q22	536	482	485	482
Total cold run time: 72488 ms
Total hot run time: 56850 ms

doris-robot · 2024-08-06T15:05:31Z

TPC-DS: Total hot run time: 169071 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 430be09f8d87842502c150362971b65ca30b1c42, data reload: false
query1	907	371	374	371
query2	6451	1678	1648	1648
query3	6673	219	216	216
query4	20043	17508	17247	17247
query5	3636	509	501	501
query6	275	177	166	166
query7	4600	306	305	305
query8	244	198	199	198
query9	8514	2382	2362	2362
query10	429	286	265	265
query11	10427	10049	9976	9976
query12	123	91	89	89
query13	1638	378	378	378
query14	9730	7443	6722	6722
query15	212	169	160	160
query16	6945	475	432	432
query17	952	569	566	566
query18	1923	289	291	289
query19	192	145	155	145
query20	95	87	88	87
query21	209	98	103	98
query22	4146	4027	4063	4027
query23	33964	33355	33404	33355
query24	9309	3188	3172	3172
query25	678	401	410	401
query26	1395	151	151	151
query27	3027	279	280	279
query28	7617	2028	2026	2026
query29	1037	440	425	425
query30	239	153	151	151
query31	925	776	791	776
query32	100	53	61	53
query33	687	328	334	328
query34	922	505	508	505
query35	891	770	749	749
query36	1038	885	884	884
query37	164	81	86	81
query38	2925	2848	2763	2763
query39	855	809	820	809
query40	253	114	108	108
query41	44	43	43	43
query42	119	103	102	102
query43	462	410	407	407
query44	1194	735	756	735
query45	208	175	177	175
query46	1084	832	786	786
query47	1806	1674	1745	1674
query48	362	289	290	289
query49	877	423	430	423
query50	891	435	443	435
query51	6691	6680	6674	6674
query52	112	88	92	88
query53	260	182	180	180
query54	629	456	447	447
query55	77	75	77	75
query56	285	248	259	248
query57	1116	1029	1064	1029
query58	273	258	267	258
query59	2492	2349	2347	2347
query60	291	278	273	273
query61	100	100	95	95
query62	880	650	650	650
query63	218	186	185	185
query64	5088	1914	1871	1871
query65	3153	3083	3120	3083
query66	979	324	338	324
query67	15331	14931	14916	14916
query68	4318	576	580	576
query69	591	376	304	304
query70	1076	1014	1138	1014
query71	379	289	285	285
query72	7057	2732	2543	2543
query73	780	330	332	330
query74	6015	5612	5562	5562
query75	3391	2742	2738	2738
query76	2105	1227	1284	1227
query77	573	304	307	304
query78	9357	8973	8832	8832
query79	2335	535	537	535
query80	1169	501	499	499
query81	584	230	223	223
query82	798	137	131	131
query83	254	167	167	167
query84	258	81	79	79
query85	1523	321	303	303
query86	484	300	306	300
query87	3243	3070	3049	3049
query88	3847	2444	2423	2423
query89	391	291	293	291
query90	1786	195	199	195
query91	130	104	105	104
query92	57	54	49	49
query93	2279	609	630	609
query94	822	301	293	293
query95	383	269	273	269
query96	612	283	290	283
query97	3265	3031	3042	3031
query98	222	209	196	196
query99	1642	1288	1251	1251
Total cold run time: 261138 ms
Total hot run time: 169071 ms

doris-robot · 2024-08-06T15:11:01Z

ClickBench: Total hot run time: 30.45 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 430be09f8d87842502c150362971b65ca30b1c42, data reload: false
query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.22	0.04	0.04
query4	1.69	0.06	0.07
query5	0.49	0.49	0.48
query6	1.14	0.72	0.70
query7	0.02	0.01	0.02
query8	0.06	0.04	0.05
query9	0.57	0.51	0.51
query10	0.57	0.56	0.56
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.62	0.62	0.60
query14	0.77	0.79	0.80
query15	0.89	0.86	0.86
query16	0.35	0.36	0.35
query17	0.99	0.99	1.01
query18	0.21	0.21	0.21
query19	1.84	1.70	1.71
query20	0.01	0.01	0.02
query21	15.44	0.75	0.65
query22	4.29	7.45	1.64
query23	17.97	1.44	1.40
query24	2.28	0.22	0.22
query25	0.19	0.08	0.07
query26	0.32	0.22	0.22
query27	0.46	0.23	0.23
query28	13.17	1.01	0.98
query29	12.58	3.37	3.34
query30	0.25	0.06	0.06
query31	2.86	0.41	0.41
query32	3.24	0.48	0.50
query33	2.95	2.94	2.96
query34	15.46	4.27	4.24
query35	4.30	4.29	4.29
query36	0.68	0.47	0.48
query37	0.18	0.17	0.16
query38	0.16	0.15	0.15
query39	0.04	0.04	0.03
query40	0.16	0.13	0.12
query41	0.10	0.05	0.04
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 108.05 s
Total hot run time: 30.45 s

dataroaring

LGTM

zclllyybb · 2024-08-07T02:16:25Z

run buildall

github-actions · 2024-08-07T02:23:14Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-08-07T03:35:52Z

TPC-H: Total hot run time: 41997 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 336d948b6a8dbf72b137c9cb3646f12bda25b25e, data reload: false
------ Round 1 ----------------------------------
q1	17701	4234	4165	4165
q2	2037	204	201	201
q3	10559	1315	1362	1315
q4	10170	823	949	823
q5	7665	3065	3020	3020
q6	225	146	144	144
q7	1064	624	618	618
q8	9435	1958	1980	1958
q9	8583	6611	6605	6605
q10	8785	3884	3861	3861
q11	432	252	259	252
q12	418	230	229	229
q13	17778	2959	2963	2959
q14	273	239	248	239
q15	525	491	505	491
q16	526	400	386	386
q17	1003	937	922	922
q18	8040	7338	7219	7219
q19	1638	1238	1226	1226
q20	564	315	325	315
q21	5289	4755	4772	4755
q22	355	294	294	294
Total cold run time: 113065 ms
Total hot run time: 41997 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4168	4122	4034	4034
q2	338	225	221	221
q3	3043	3045	3196	3045
q4	2016	2077	1979	1979
q5	5635	5482	5467	5467
q6	235	130	133	130
q7	2142	1756	1830	1756
q8	3323	3375	3380	3375
q9	8748	8702	8870	8702
q10	3934	4097	3909	3909
q11	551	450	458	450
q12	774	623	592	592
q13	16323	3132	3160	3132
q14	293	269	278	269
q15	536	486	501	486
q16	456	423	418	418
q17	1791	1761	1767	1761
q18	8203	7699	7653	7653
q19	1746	1767	1745	1745
q20	2038	1917	1850	1850
q21	5816	5537	5328	5328
q22	512	482	486	482
Total cold run time: 72621 ms
Total hot run time: 56784 ms

doris-robot · 2024-08-07T03:46:32Z

TPC-DS: Total hot run time: 168626 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 336d948b6a8dbf72b137c9cb3646f12bda25b25e, data reload: false
query1	924	383	370	370
query2	6455	1704	1635	1635
query3	6671	216	225	216
query4	20001	17088	17441	17088
query5	3626	530	514	514
query6	271	170	181	170
query7	4592	304	289	289
query8	246	195	201	195
query9	8519	2364	2349	2349
query10	428	289	273	273
query11	10522	10079	9903	9903
query12	123	94	89	89
query13	1649	397	379	379
query14	9987	6678	7238	6678
query15	205	170	171	170
query16	6786	452	454	452
query17	990	575	566	566
query18	1846	287	297	287
query19	197	152	148	148
query20	96	89	88	88
query21	204	104	99	99
query22	4358	4162	4018	4018
query23	33804	33738	33479	33479
query24	10195	3015	3116	3015
query25	689	409	406	406
query26	1788	159	155	155
query27	2993	288	283	283
query28	7549	2001	1986	1986
query29	1241	429	430	429
query30	232	156	161	156
query31	980	784	774	774
query32	109	60	62	60
query33	708	344	344	344
query34	943	529	510	510
query35	843	759	718	718
query36	1034	897	903	897
query37	284	79	84	79
query38	2893	2739	2746	2739
query39	886	821	822	821
query40	252	112	112	112
query41	45	44	44	44
query42	118	98	98	98
query43	450	422	421	421
query44	1164	729	731	729
query45	211	176	179	176
query46	1078	822	782	782
query47	1800	1723	1734	1723
query48	364	291	301	291
query49	951	417	432	417
query50	914	431	450	431
query51	6798	6703	6631	6631
query52	111	86	92	86
query53	251	178	180	178
query54	634	452	447	447
query55	78	74	76	74
query56	309	274	264	264
query57	1144	1048	1055	1048
query58	270	273	257	257
query59	2597	2326	2370	2326
query60	319	264	278	264
query61	96	95	101	95
query62	903	652	660	652
query63	212	184	187	184
query64	5560	1889	1904	1889
query65	3152	3130	3099	3099
query66	1296	323	325	323
query67	15288	14803	14773	14773
query68	4415	563	573	563
query69	452	308	300	300
query70	1098	1074	1054	1054
query71	456	283	277	277
query72	7118	2667	2458	2458
query73	766	332	334	332
query74	6062	5632	5647	5632
query75	3416	2724	2707	2707
query76	2791	1180	1285	1180
query77	428	311	323	311
query78	9383	8825	8829	8825
query79	1970	538	537	537
query80	1907	512	505	505
query81	567	224	229	224
query82	988	140	132	132
query83	274	174	171	171
query84	270	81	81	81
query85	1248	331	304	304
query86	397	318	296	296
query87	3254	3051	3138	3051
query88	3055	2423	2428	2423
query89	390	295	289	289
query90	1784	200	195	195
query91	126	99	98	98
query92	60	49	53	49
query93	1569	624	615	615
query94	816	293	295	293
query95	379	275	281	275
query96	603	281	283	281
query97	3232	3070	3055	3055
query98	219	206	199	199
query99	1611	1280	1273	1273
Total cold run time: 263336 ms
Total hot run time: 168626 ms

doris-robot · 2024-08-07T03:52:02Z

ClickBench: Total hot run time: 30.12 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 336d948b6a8dbf72b137c9cb3646f12bda25b25e, data reload: false
query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.67	0.07	0.07
query5	0.48	0.49	0.47
query6	1.11	0.72	0.71
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.57	0.52	0.50
query10	0.56	0.55	0.55
query11	0.15	0.12	0.12
query12	0.16	0.12	0.12
query13	0.61	0.60	0.60
query14	0.77	0.80	0.79
query15	0.89	0.86	0.86
query16	0.36	0.35	0.36
query17	1.02	0.99	1.00
query18	0.23	0.22	0.22
query19	1.94	1.74	1.76
query20	0.02	0.01	0.01
query21	15.40	0.74	0.67
query22	4.08	7.58	1.27
query23	17.86	1.33	1.34
query24	2.32	0.22	0.21
query25	0.19	0.08	0.08
query26	0.31	0.21	0.21
query27	0.46	0.23	0.24
query28	13.16	1.00	0.96
query29	12.50	3.30	3.32
query30	0.25	0.06	0.06
query31	2.86	0.40	0.40
query32	3.25	0.49	0.48
query33	2.95	2.96	3.00
query34	15.51	4.31	4.31
query35	4.30	4.31	4.30
query36	0.69	0.48	0.49
query37	0.19	0.16	0.16
query38	0.16	0.14	0.14
query39	0.04	0.03	0.04
query40	0.16	0.13	0.13
query41	0.09	0.05	0.05
query42	0.05	0.05	0.04
query43	0.05	0.04	0.05
Total cold run time: 107.78 s
Total hot run time: 30.12 s

HappenLee

LGTM

github-actions · 2024-08-07T09:44:00Z

PR approved by at least one committer and no changes requested.

…hange some Errors stacktrace (#38978) ## Proposed changes Issue Number: close #xxx 1. introduce `ERROR_CTOR_NOSTACK` to make some Doris' native error don't print stacktrace by default. 2. when streamload exceed limit, just tell user which conf to change

…hange some Errors stacktrace (apache#38978) ## Proposed changes Issue Number: close #xxx 1. introduce `ERROR_CTOR_NOSTACK` to make some Doris' native error don't print stacktrace by default. 2. when streamload exceed limit, just tell user which conf to change

…hange some Errors stacktrace (#38978) ## Proposed changes Issue Number: close #xxx 1. introduce `ERROR_CTOR_NOSTACK` to make some Doris' native error don't print stacktrace by default. 2. when streamload exceed limit, just tell user which conf to change

…hange some Errors stacktrace (apache#38978) Issue Number: close #xxx 1. introduce `ERROR_CTOR_NOSTACK` to make some Doris' native error don't print stacktrace by default. 2. when streamload exceed limit, just tell user which conf to change

## Proposed changes Issue Number: close #xxx #38385 optimize parsing datetime #38978 make stream load failure message more clear and disable some error's stacktrace by default #39255 fix random function coredump #39324 fix function corr inconsistency with doc #39449 check auto partitoin nullity when creating partition #39695 make DynamicPartitionScheduler immediately know interval's change #39754 Add some partition expr check on creating table

…hange some Errors stacktrace (apache#38978) ## Proposed changes Issue Number: close #xxx 1. introduce `ERROR_CTOR_NOSTACK` to make some Doris' native error don't print stacktrace by default. 2. when streamload exceed limit, just tell user which conf to change

github-actionsBot added the doing label Aug 6, 2024

zclllyybb force-pushed the errmsg branch from 9eafd85 to 430be09Compare August 6, 2024 13:19

yiguolei previously approved these changes Aug 6, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Aug 6, 2024

github-actionsBot added the reviewed label Aug 6, 2024

dataroaring previously approved these changes Aug 7, 2024
View reviewed changes

1
336d948

zclllyybb dismissed stale reviews from dataroaring and yiguolei via 336d948August 7, 2024 02:16

zclllyybb force-pushed the errmsg branch from 430be09 to 336d948Compare August 7, 2024 02:16

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Aug 7, 2024

HappenLee approved these changes Aug 7, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Aug 7, 2024

HappenLee added the dev/2.1.x label Aug 7, 2024

zhangstar333 approved these changes Aug 7, 2024
View reviewed changes

zhangstar333 merged commit 4e0c3ce into apache:masterAug 7, 2024

zhangstar333 added the dev/3.0.x label Aug 7, 2024

dataroaring added dev/3.0.2-merged and removed dev/3.0.x labels Aug 12, 2024

zclllyybb deleted the errmsg branch August 23, 2024 09:57

zclllyybb mentioned this pull request Aug 23, 2024
[branch-2.1] pick some prs #39860
Merged

yiguolei added dev/2.1.6-merged and removed dev/2.1.x labels Aug 24, 2024

Uh oh!

Conversation

zclllyybb commented Aug 6, 2024

Proposed changes

Uh oh!

doris-robot commented Aug 6, 2024

Uh oh!

zclllyybb commented Aug 6, 2024

Uh oh!

github-actionsBot commented Aug 6, 2024

Uh oh!

github-actionsBot commented Aug 6, 2024

Uh oh!

github-actionsBot commented Aug 6, 2024

Uh oh!

github-actionsBot commented Aug 6, 2024

Uh oh!

doris-robot commented Aug 6, 2024

Uh oh!

doris-robot commented Aug 6, 2024

Uh oh!

doris-robot commented Aug 6, 2024

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

zclllyybb commented Aug 7, 2024

Uh oh!

github-actionsBot commented Aug 7, 2024

Uh oh!

doris-robot commented Aug 7, 2024

Uh oh!

doris-robot commented Aug 7, 2024

Uh oh!

doris-robot commented Aug 7, 2024

Uh oh!

HappenLee left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Aug 7, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants