[fix](routine-load) fix show routine load job result is incorrect by sollhui · Pull Request #38199 · apache/doris

sollhui · 2024-07-22T07:05:52Z

Use the sql to query routine load records:

show routine load for db.job1\G

return all routine load job in the db:

10 rows in set (0.02 sec)

why the bug happen is there is no correct assignment when analyze the sql:

if (Strings.isNullOrEmpty(dbName)) {
dbFullName = analyzer.getContext().getDatabase();
if (Strings.isNullOrEmpty(dbFullName)) {
ErrorReport.reportAnalysisException(ErrorCode.ERR_NO_DB_ERROR);
}
}

dbFullName will always null if dbName is not null or empty.

The bug is introduce by: #27861

doris-robot · 2024-07-22T07:05:57Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

sollhui · 2024-07-22T07:06:31Z

run buildall

sollhui · 2024-07-22T07:08:07Z

run buildall

doris-robot · 2024-07-22T07:40:42Z

TPC-H: Total hot run time: 39736 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 16048f3e8b154149430dd713d151c4e169b82cc3, data reload: false
------ Round 1 ----------------------------------
q1	17618	4339	4290	4290
q2	2006	194	195	194
q3	10460	1184	1059	1059
q4	10190	778	846	778
q5	7632	2704	2636	2636
q6	224	142	141	141
q7	956	608	616	608
q8	9235	2057	2070	2057
q9	8914	6528	6524	6524
q10	8811	3784	3769	3769
q11	465	247	230	230
q12	418	231	226	226
q13	18922	3015	2980	2980
q14	275	229	247	229
q15	526	473	490	473
q16	502	373	376	373
q17	971	665	713	665
q18	8063	7373	7335	7335
q19	7069	1479	1419	1419
q20	663	308	317	308
q21	4899	3275	3158	3158
q22	354	285	284	284
Total cold run time: 119173 ms
Total hot run time: 39736 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4474	4295	4248	4248
q2	374	269	266	266
q3	3019	2909	2939	2909
q4	2005	1733	1787	1733
q5	5599	5559	5474	5474
q6	222	134	135	134
q7	2301	1893	1861	1861
q8	3263	3449	3381	3381
q9	8807	8890	8804	8804
q10	4149	3764	3864	3764
q11	629	503	503	503
q12	797	633	638	633
q13	15958	3192	3203	3192
q14	319	286	292	286
q15	517	505	476	476
q16	489	437	441	437
q17	1818	1551	1540	1540
q18	8159	8063	7728	7728
q19	1736	1620	1525	1525
q20	2116	1873	1894	1873
q21	7534	4986	4703	4703
q22	622	569	508	508
Total cold run time: 74907 ms
Total hot run time: 55978 ms

doris-robot · 2024-07-22T07:51:41Z

TPC-DS: Total hot run time: 174246 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 16048f3e8b154149430dd713d151c4e169b82cc3, data reload: false
query1	919	384	372	372
query2	6422	1873	1889	1873
query3	6659	208	215	208
query4	20133	17634	17459	17459
query5	3701	478	486	478
query6	290	170	161	161
query7	4579	299	284	284
query8	231	185	188	185
query9	8707	2442	2405	2405
query10	451	305	272	272
query11	11644	10039	10003	10003
query12	123	102	86	86
query13	1662	373	378	373
query14	10144	8536	7908	7908
query15	223	173	169	169
query16	7271	499	475	475
query17	1594	577	562	562
query18	1230	287	285	285
query19	211	151	160	151
query20	93	131	82	82
query21	203	128	127	127
query22	4280	4058	4152	4058
query23	33934	33710	33684	33684
query24	11120	2895	2975	2895
query25	603	389	387	387
query26	708	158	151	151
query27	2227	281	273	273
query28	5891	2044	2038	2038
query29	905	623	645	623
query30	246	164	154	154
query31	989	763	792	763
query32	97	56	55	55
query33	742	338	330	330
query34	902	486	510	486
query35	866	725	764	725
query36	1137	967	959	959
query37	147	85	82	82
query38	2981	2862	2787	2787
query39	900	884	862	862
query40	205	122	120	120
query41	50	43	44	43
query42	120	108	100	100
query43	526	477	460	460
query44	1118	753	721	721
query45	193	161	162	161
query46	1089	733	694	694
query47	1849	1745	1754	1745
query48	363	295	288	288
query49	828	407	424	407
query50	776	398	386	386
query51	6783	6621	6638	6621
query52	103	97	90	90
query53	373	290	295	290
query54	955	449	449	449
query55	74	76	72	72
query56	286	264	273	264
query57	1124	1061	1117	1061
query58	249	277	276	276
query59	2849	2744	2526	2526
query60	317	288	283	283
query61	96	95	92	92
query62	791	630	663	630
query63	327	291	295	291
query64	9141	2231	1669	1669
query65	3212	3136	3110	3110
query66	776	328	325	325
query67	15429	15128	15014	15014
query68	4466	537	537	537
query69	522	383	366	366
query70	1181	1093	1148	1093
query71	410	284	284	284
query72	6910	5395	5793	5395
query73	744	321	326	321
query74	6045	5744	5645	5645
query75	3395	2695	2725	2695
query76	2772	942	911	911
query77	436	325	312	312
query78	9672	9051	8960	8960
query79	2497	526	515	515
query80	2728	459	475	459
query81	603	219	224	219
query82	760	144	134	134
query83	307	167	164	164
query84	271	90	85	85
query85	2095	372	300	300
query86	484	330	313	313
query87	3295	3092	3089	3089
query88	4168	2465	2468	2465
query89	487	384	403	384
query90	1866	199	186	186
query91	127	101	100	100
query92	64	50	48	48
query93	2253	505	499	499
query94	1116	290	297	290
query95	405	318	317	317
query96	597	277	276	276
query97	3164	3023	3060	3023
query98	218	199	193	193
query99	1622	1250	1280	1250
Total cold run time: 270847 ms
Total hot run time: 174246 ms

doris-robot · 2024-07-22T07:57:13Z

ClickBench: Total hot run time: 30.32 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 16048f3e8b154149430dd713d151c4e169b82cc3, data reload: false
query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.06	0.06
query4	1.65	0.09	0.08
query5	0.49	0.49	0.50
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.06	0.05	0.04
query9	0.55	0.49	0.47
query10	0.53	0.55	0.55
query11	0.14	0.11	0.11
query12	0.14	0.12	0.13
query13	0.60	0.59	0.58
query14	0.78	0.78	0.77
query15	0.85	0.81	0.82
query16	0.35	0.35	0.37
query17	0.95	1.02	0.97
query18	0.23	0.22	0.22
query19	1.78	1.70	1.71
query20	0.01	0.01	0.02
query21	15.41	0.75	0.63
query22	4.08	7.40	1.68
query23	18.31	1.36	1.27
query24	2.16	0.22	0.22
query25	0.17	0.09	0.08
query26	0.30	0.22	0.21
query27	0.46	0.23	0.23
query28	13.23	1.02	0.98
query29	12.60	3.43	3.39
query30	0.25	0.06	0.05
query31	2.86	0.41	0.38
query32	3.27	0.48	0.48
query33	2.93	2.86	3.00
query34	16.92	4.34	4.32
query35	4.44	4.38	4.43
query36	0.65	0.45	0.47
query37	0.18	0.17	0.15
query38	0.16	0.15	0.14
query39	0.05	0.03	0.04
query40	0.14	0.12	0.13
query41	0.08	0.04	0.05
query42	0.05	0.04	0.04
query43	0.04	0.05	0.04
Total cold run time: 109.35 s
Total hot run time: 30.32 s

dataroaring

LGTM

github-actions · 2024-07-22T09:03:03Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-07-22T09:03:05Z

PR approved by anyone and no changes requested.

sollhui · 2024-07-22T09:14:00Z

run buildall

sollhui · 2024-07-22T09:15:15Z

run buildall

dataroaring

LGTM

github-actions · 2024-07-22T09:18:51Z

PR approved by at least one committer and no changes requested.

liaoxin01

LGTM

doris-robot · 2024-07-22T10:06:28Z

TPC-H: Total hot run time: 40266 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 90f0795997c61fef997eb08b36b33c7c7b38f524, data reload: false
------ Round 1 ----------------------------------
q1	18176	4552	4432	4432
q2	2503	198	199	198
q3	11645	1206	1190	1190
q4	10665	814	886	814
q5	7720	2750	2754	2750
q6	228	141	140	140
q7	997	616	617	616
q8	9489	2099	2116	2099
q9	8827	6604	6617	6604
q10	8898	3784	3743	3743
q11	455	240	242	240
q12	393	234	219	219
q13	18774	2993	2988	2988
q14	277	237	242	237
q15	528	491	505	491
q16	519	376	377	376
q17	977	772	698	698
q18	8194	7522	7347	7347
q19	4984	1395	1317	1317
q20	663	323	325	323
q21	4893	3165	3260	3165
q22	354	281	279	279
Total cold run time: 120159 ms
Total hot run time: 40266 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4385	4234	4308	4234
q2	380	260	265	260
q3	2999	2767	2759	2759
q4	1888	1609	1632	1609
q5	5355	5328	5308	5308
q6	224	132	132	132
q7	2125	1738	1695	1695
q8	3195	3382	3292	3292
q9	8364	8426	8370	8370
q10	3928	3668	3685	3668
q11	599	478	485	478
q12	764	615	623	615
q13	16833	2991	2984	2984
q14	296	285	276	276
q15	507	476	475	475
q16	474	406	449	406
q17	1771	1480	1488	1480
q18	7745	7485	7379	7379
q19	1669	1527	1331	1331
q20	1985	1780	1803	1780
q21	4957	4709	4613	4613
q22	584	489	480	480
Total cold run time: 71027 ms
Total hot run time: 53624 ms

doris-robot · 2024-07-22T10:17:47Z

TPC-DS: Total hot run time: 173405 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 90f0795997c61fef997eb08b36b33c7c7b38f524, data reload: false
query1	906	372	375	372
query2	6455	1952	1986	1952
query3	6665	207	215	207
query4	28441	17725	17267	17267
query5	4177	478	475	475
query6	248	177	164	164
query7	4594	286	280	280
query8	237	189	191	189
query9	8699	2477	2440	2440
query10	454	298	288	288
query11	10703	10139	10225	10139
query12	129	85	80	80
query13	1637	364	358	358
query14	10094	7564	7795	7564
query15	231	162	169	162
query16	7825	466	444	444
query17	1203	536	522	522
query18	1975	274	277	274
query19	191	146	141	141
query20	88	82	79	79
query21	211	124	124	124
query22	4279	3992	3906	3906
query23	33825	33099	33128	33099
query24	12318	2813	2902	2813
query25	665	368	378	368
query26	1792	153	150	150
query27	2968	271	274	271
query28	7612	2002	1992	1992
query29	1188	616	604	604
query30	283	147	146	146
query31	978	747	741	741
query32	94	51	52	51
query33	771	334	324	324
query34	931	471	490	471
query35	856	739	718	718
query36	1110	930	939	930
query37	278	81	77	77
query38	2893	2765	2777	2765
query39	877	793	823	793
query40	273	119	119	119
query41	48	50	41	41
query42	117	97	95	95
query43	548	479	478	478
query44	1210	713	710	710
query45	192	161	163	161
query46	1102	742	744	742
query47	1849	1769	1757	1757
query48	376	287	320	287
query49	1218	402	404	402
query50	784	404	394	394
query51	6770	6639	6603	6603
query52	105	88	91	88
query53	355	288	283	283
query54	1015	448	454	448
query55	77	74	73	73
query56	289	272	275	272
query57	1175	1064	1056	1056
query58	258	263	252	252
query59	2932	2591	2598	2591
query60	310	289	266	266
query61	96	98	95	95
query62	841	642	692	642
query63	321	289	288	288
query64	10461	2204	7433	2204
query65	3117	3116	3118	3116
query66	1363	324	344	324
query67	15387	14927	14831	14831
query68	4507	528	543	528
query69	555	399	344	344
query70	1174	1151	1149	1149
query71	408	284	282	282
query72	7481	5750	5494	5494
query73	755	327	328	327
query74	6319	5766	5719	5719
query75	3570	2676	2677	2676
query76	2736	934	918	918
query77	683	311	309	309
query78	9719	9007	9627	9007
query79	2118	519	515	515
query80	1781	475	482	475
query81	569	219	220	219
query82	700	132	130	130
query83	330	162	165	162
query84	273	88	85	85
query85	1636	322	336	322
query86	478	326	328	326
query87	3375	3158	3120	3120
query88	4076	2480	2439	2439
query89	476	390	380	380
query90	1830	189	187	187
query91	128	103	100	100
query92	66	53	51	51
query93	2163	499	487	487
query94	1251	250	283	250
query95	407	312	319	312
query96	602	278	274	274
query97	3187	3015	2995	2995
query98	221	203	193	193
query99	1641	1274	1272	1272
Total cold run time: 286685 ms
Total hot run time: 173405 ms

doris-robot · 2024-07-22T10:23:17Z

ClickBench: Total hot run time: 30.73 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 90f0795997c61fef997eb08b36b33c7c7b38f524, data reload: false
query1	0.04	0.04	0.03
query2	0.08	0.05	0.04
query3	0.23	0.05	0.05
query4	1.67	0.08	0.08
query5	0.50	0.49	0.49
query6	1.15	0.72	0.74
query7	0.03	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.54	0.55	0.55
query11	0.15	0.11	0.11
query12	0.16	0.12	0.12
query13	0.60	0.59	0.60
query14	0.75	0.78	0.78
query15	0.86	0.82	0.80
query16	0.37	0.37	0.35
query17	1.00	0.95	0.94
query18	0.22	0.22	0.22
query19	1.75	1.80	1.74
query20	0.01	0.01	0.00
query21	15.39	0.78	0.65
query22	4.17	6.98	2.03
query23	18.30	1.41	1.24
query24	2.19	0.22	0.23
query25	0.14	0.09	0.09
query26	0.29	0.20	0.21
query27	0.45	0.24	0.22
query28	13.20	1.02	0.99
query29	12.63	3.32	3.30
query30	0.25	0.06	0.06
query31	2.87	0.40	0.39
query32	3.35	0.47	0.48
query33	2.83	2.92	2.96
query34	16.97	4.34	4.35
query35	4.43	4.39	4.41
query36	0.66	0.47	0.47
query37	0.19	0.16	0.16
query38	0.15	0.15	0.15
query39	0.04	0.04	0.03
query40	0.15	0.13	0.12
query41	0.10	0.05	0.05
query42	0.06	0.05	0.06
query43	0.05	0.04	0.04
Total cold run time: 109.57 s
Total hot run time: 30.73 s

…8199) Use the sql to query routine load records: ``` show routine load for db.job1\G ``` return all routine load job in the db: ``` 10 rows in set (0.02 sec) ``` why the bug happen is there is no correct assignment when analyze the sql: ``` if (Strings.isNullOrEmpty(dbName)) { dbFullName = analyzer.getContext().getDatabase(); if (Strings.isNullOrEmpty(dbFullName)) { ErrorReport.reportAnalysisException(ErrorCode.ERR_NO_DB_ERROR); } } ``` dbFullName will always null if dbName is not null or empty. The bug is introduce by: #27861

…ache#38199) Use the sql to query routine load records: ``` show routine load for db.job1\G ``` return all routine load job in the db: ``` 10 rows in set (0.02 sec) ``` why the bug happen is there is no correct assignment when analyze the sql: ``` if (Strings.isNullOrEmpty(dbName)) { dbFullName = analyzer.getContext().getDatabase(); if (Strings.isNullOrEmpty(dbFullName)) { ErrorReport.reportAnalysisException(ErrorCode.ERR_NO_DB_ERROR); } } ``` dbFullName will always null if dbName is not null or empty. The bug is introduce by: apache#27861

dataroaring previously approved these changes Jul 22, 2024
View reviewed changes

dataroaring added dev/2.1.x dev/3.0.x labels Jul 22, 2024

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jul 22, 2024

github-actionsBot added the reviewed label Jul 22, 2024

sollhui dismissed dataroaring’s stale review via fc4c6bfJuly 22, 2024 09:13

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Jul 22, 2024

fix show routine load job result is incorrect
90f0795

dataroaring approved these changes Jul 22, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jul 22, 2024

liaoxin01 approved these changes Jul 22, 2024
View reviewed changes

dataroaring merged commit e97ebd6 into apache:masterJul 23, 2024

dataroaring added dev/3.0.1-merged and removed dev/3.0.x labels Jul 23, 2024

sollhui mentioned this pull request Aug 4, 2024
[fix](routine-load) fix show routine load job result is incorrect (#38199) #38823
Closed

sollhui mentioned this pull request Aug 4, 2024
[fix](routine-load) fix show routine load job result is incorrect (#38199) #38827
Closed

liaoxin01 removed the dev/2.1.x label Aug 5, 2024

gavinchou mentioned this pull request Aug 19, 2024
Release Note 3.0.1 #39570
Closed

Uh oh!

Conversation

sollhui commented Jul 22, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

sollhui commented Jul 22, 2024

Uh oh!

sollhui commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jul 22, 2024

Uh oh!

github-actionsBot commented Jul 22, 2024

Uh oh!

sollhui commented Jul 22, 2024

Uh oh!

sollhui commented Jul 22, 2024

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jul 22, 2024

Uh oh!

liaoxin01 left a comment

Choose a reason for hiding this comment

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

doris-robot commented Jul 22, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sollhui commented Jul 22, 2024•
edited
Loading