[fix](hive) support null_format and escape_char for hive text by suxiaogang223 · Pull Request #40291 · apache/doris

suxiaogang223 · 2024-09-02T17:02:50Z

Proposed changes

related pr: #39700
support escape.delim and serialization.null.format for hive text

doris-robot · 2024-09-02T17:02:54Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

github-actions · 2024-09-02T17:03:35Z

Possible file(s) that should be tracked in LFS detected: 🚨

The following file(s) exceeds the file size limit: 1048576 bytes, as set in the .yml configuration files:

regression-test/data/external_table_p0/hive/test_hive_basic_type.out

Consider using git-lfs to manage large files.

github-actions · 2024-09-02T17:09:50Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-09-02T17:18:33Z

Possible file(s) that should be tracked in LFS detected: 🚨

The following file(s) exceeds the file size limit: 1048576 bytes, as set in the .yml configuration files:

regression-test/data/external_table_p0/hive/test_hive_basic_type.out

Consider using git-lfs to manage large files.

suxiaogang223 · 2024-09-02T17:20:33Z

run buildall

github-actions · 2024-09-02T17:24:59Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-09-02T19:17:36Z

TPC-H: Total hot run time: 38242 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bedec46f1f3632079a88b2c876d0c3ecad473e0d, data reload: false
------ Round 1 ----------------------------------
q1	18619	4955	4368	4368
q2	2040	182	174	174
q3	11455	959	1090	959
q4	10236	731	751	731
q5	7763	2875	2836	2836
q6	228	141	137	137
q7	954	611	602	602
q8	9324	2073	2075	2073
q9	7239	6560	6631	6560
q10	7008	2194	2199	2194
q11	460	239	240	239
q12	398	225	226	225
q13	18948	3074	3101	3074
q14	288	247	233	233
q15	514	488	507	488
q16	599	506	507	506
q17	1005	686	712	686
q18	7613	6789	6915	6789
q19	1401	1097	963	963
q20	669	330	322	322
q21	3963	3099	3126	3099
q22	1113	1017	984	984
Total cold run time: 111837 ms
Total hot run time: 38242 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4352	4340	4325	4325
q2	381	281	269	269
q3	2865	2641	2690	2641
q4	1885	1634	1684	1634
q5	5685	5637	5817	5637
q6	231	136	137	136
q7	2210	1837	1789	1789
q8	3250	3412	3498	3412
q9	8821	8780	8847	8780
q10	3545	3407	3341	3341
q11	612	508	499	499
q12	851	668	663	663
q13	14063	3093	3306	3093
q14	320	283	290	283
q15	528	489	494	489
q16	615	571	601	571
q17	1843	1550	1553	1550
q18	8145	7832	7971	7832
q19	1731	1552	1683	1552
q20	2140	1886	1925	1886
q21	5730	5511	5406	5406
q22	1121	1016	994	994
Total cold run time: 70924 ms
Total hot run time: 56782 ms

doris-robot · 2024-09-02T19:29:50Z

TPC-DS: Total hot run time: 191827 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bedec46f1f3632079a88b2c876d0c3ecad473e0d, data reload: false
query1	1275	890	843	843
query2	6294	1943	1879	1879
query3	10594	4011	3925	3925
query4	59770	23665	23082	23082
query5	5392	511	509	509
query6	403	162	189	162
query7	5773	295	290	290
query8	286	199	208	199
query9	8932	2456	2447	2447
query10	500	268	266	266
query11	18182	14897	15134	14897
query12	179	109	111	109
query13	1562	396	381	381
query14	10830	7122	6731	6731
query15	242	188	180	180
query16	7551	492	488	488
query17	1112	596	575	575
query18	2038	319	312	312
query19	295	154	155	154
query20	124	106	109	106
query21	203	107	104	104
query22	4525	4360	4440	4360
query23	34198	33565	33217	33217
query24	6017	2873	2869	2869
query25	552	412	410	410
query26	684	167	159	159
query27	1758	282	279	279
query28	3779	2069	2052	2052
query29	697	434	430	430
query30	244	152	152	152
query31	933	716	745	716
query32	87	54	54	54
query33	466	297	289	289
query34	862	479	473	473
query35	858	724	735	724
query36	1075	959	936	936
query37	156	93	92	92
query38	4006	3852	3917	3852
query39	1450	1427	1404	1404
query40	209	122	119	119
query41	49	48	48	48
query42	113	96	99	96
query43	497	462	467	462
query44	1100	740	744	740
query45	198	164	166	164
query46	1090	746	735	735
query47	1871	1799	1820	1799
query48	375	302	303	302
query49	787	437	458	437
query50	818	414	422	414
query51	7003	6912	6899	6899
query52	100	89	89	89
query53	255	185	181	181
query54	581	464	460	460
query55	77	79	78	78
query56	287	268	277	268
query57	1232	1055	1051	1051
query58	229	277	222	222
query59	2938	2819	2880	2819
query60	301	282	288	282
query61	124	119	122	119
query62	749	651	675	651
query63	218	187	187	187
query64	2650	809	690	690
query65	3258	3185	3115	3115
query66	651	350	339	339
query67	15441	15191	15253	15191
query68	4442	564	574	564
query69	419	287	281	281
query70	1200	1127	1156	1127
query71	415	285	276	276
query72	6608	4005	3983	3983
query73	778	331	328	328
query74	9143	8915	8880	8880
query75	3355	2624	2705	2624
query76	1787	1028	1088	1028
query77	587	328	308	308
query78	10587	9401	9195	9195
query79	1626	537	544	537
query80	909	514	524	514
query81	564	238	239	238
query82	252	144	149	144
query83	184	152	145	145
query84	263	77	73	73
query85	1043	324	284	284
query86	379	311	284	284
query87	4632	4186	4286	4186
query88	3626	2294	2312	2294
query89	393	279	287	279
query90	1761	195	192	192
query91	126	98	97	97
query92	65	48	50	48
query93	1372	546	550	546
query94	710	288	297	288
query95	354	258	257	257
query96	591	267	262	262
query97	3231	3099	3044	3044
query98	224	209	195	195
query99	1545	1309	1262	1262
Total cold run time: 310256 ms
Total hot run time: 191827 ms

doris-robot · 2024-09-02T19:35:28Z

ClickBench: Total hot run time: 32.38 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bedec46f1f3632079a88b2c876d0c3ecad473e0d, data reload: false
query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.08	0.08
query5	0.50	0.50	0.52
query6	1.12	0.73	0.73
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.55	0.54	0.54
query11	0.16	0.11	0.12
query12	0.15	0.11	0.11
query13	0.61	0.59	0.58
query14	2.10	2.06	2.06
query15	0.85	0.82	0.81
query16	0.37	0.36	0.37
query17	1.05	1.00	0.95
query18	0.20	0.20	0.20
query19	1.88	1.75	1.85
query20	0.01	0.01	0.01
query21	15.40	0.67	0.67
query22	4.03	6.57	2.39
query23	18.28	1.34	1.23
query24	2.11	0.22	0.22
query25	0.15	0.08	0.09
query26	0.27	0.17	0.18
query27	0.07	0.09	0.08
query28	13.20	1.02	1.01
query29	12.62	3.31	3.36
query30	0.23	0.06	0.05
query31	2.90	0.40	0.39
query32	3.23	0.48	0.47
query33	2.94	2.98	3.10
query34	16.94	4.37	4.40
query35	4.46	4.45	4.43
query36	0.65	0.48	0.47
query37	0.18	0.15	0.15
query38	0.15	0.15	0.14
query39	0.04	0.03	0.03
query40	0.16	0.13	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.05	0.04
Total cold run time: 110.43 s
Total hot run time: 32.38 s

github-actions · 2024-09-03T03:47:53Z

Possible file(s) that should be tracked in LFS detected: 🚨

The following file(s) exceeds the file size limit: 1048576 bytes, as set in the .yml configuration files:

regression-test/data/external_table_p0/hive/test_hive_basic_type.out

Consider using git-lfs to manage large files.

suxiaogang223 · 2024-09-03T03:51:01Z

run buildall

github-actions · 2024-09-03T03:54:36Z

clang-tidy review says "All clean, LGTM! 👍"

doris-robot · 2024-09-03T05:53:11Z

TPC-H: Total hot run time: 38021 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 58697ef0bf55fe752631b3303c0cb84510bbd2cd, data reload: false
------ Round 1 ----------------------------------
q1	18434	4443	4400	4400
q2	2018	193	188	188
q3	11494	954	1098	954
q4	10295	788	686	686
q5	7755	2905	2855	2855
q6	228	134	137	134
q7	963	605	600	600
q8	9355	2090	2092	2090
q9	7531	6584	6567	6567
q10	6998	2231	2176	2176
q11	445	246	248	246
q12	403	222	217	217
q13	18935	3117	3114	3114
q14	278	235	242	235
q15	527	486	510	486
q16	590	535	507	507
q17	1008	683	694	683
q18	7549	6864	6735	6735
q19	1395	1068	1016	1016
q20	681	323	321	321
q21	3997	2782	2851	2782
q22	1134	1030	1029	1029
Total cold run time: 112013 ms
Total hot run time: 38021 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4377	4315	4296	4296
q2	381	282	261	261
q3	2933	2665	2691	2665
q4	1946	1697	1692	1692
q5	5739	5693	5821	5693
q6	232	130	130	130
q7	2244	1822	1807	1807
q8	3297	3484	3549	3484
q9	8903	8865	8891	8865
q10	3663	3361	3333	3333
q11	616	499	516	499
q12	815	679	664	664
q13	12810	3174	3277	3174
q14	308	285	302	285
q15	539	498	506	498
q16	652	587	577	577
q17	1848	1547	1529	1529
q18	8057	7785	7809	7785
q19	1728	1671	1589	1589
q20	2165	1909	1902	1902
q21	5748	5388	5530	5388
q22	1135	1068	1039	1039
Total cold run time: 70136 ms
Total hot run time: 57155 ms

doris-robot · 2024-09-03T06:05:17Z

TPC-DS: Total hot run time: 191717 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 58697ef0bf55fe752631b3303c0cb84510bbd2cd, data reload: false
query1	1260	891	884	884
query2	6341	1977	1875	1875
query3	10617	3837	3862	3837
query4	60116	23985	23227	23227
query5	5313	489	493	489
query6	407	160	163	160
query7	5766	293	303	293
query8	288	219	210	210
query9	8884	2519	2471	2471
query10	504	276	268	268
query11	15583	15196	15080	15080
query12	165	131	116	116
query13	1614	413	396	396
query14	10671	6405	7103	6405
query15	260	173	179	173
query16	7601	455	486	455
query17	1138	579	576	576
query18	1999	303	292	292
query19	311	147	149	147
query20	121	114	111	111
query21	206	103	99	99
query22	4710	4742	4523	4523
query23	34061	33467	34222	33467
query24	6101	2908	2863	2863
query25	508	404	405	404
query26	687	157	163	157
query27	1782	281	278	278
query28	3748	2057	2043	2043
query29	658	397	403	397
query30	245	156	160	156
query31	905	758	770	758
query32	85	48	53	48
query33	444	274	289	274
query34	853	475	468	468
query35	856	718	738	718
query36	1055	903	945	903
query37	146	83	81	81
query38	3998	3905	3926	3905
query39	1449	1544	1435	1435
query40	197	113	113	113
query41	46	44	48	44
query42	115	91	97	91
query43	505	487	464	464
query44	1068	734	746	734
query45	198	173	166	166
query46	1098	760	741	741
query47	1939	1824	1848	1824
query48	382	289	290	289
query49	758	424	423	423
query50	800	408	410	408
query51	7007	6839	6897	6839
query52	97	84	91	84
query53	251	177	180	177
query54	556	445	444	444
query55	73	77	71	71
query56	299	253	259	253
query57	1173	1074	1085	1074
query58	217	224	221	221
query59	2904	2749	2674	2674
query60	291	264	264	264
query61	101	98	102	98
query62	758	648	643	643
query63	212	175	187	175
query64	2826	670	685	670
query65	3211	3159	3134	3134
query66	672	336	327	327
query67	15364	15219	15228	15219
query68	2946	567	562	562
query69	399	279	271	271
query70	1169	1068	1107	1068
query71	356	270	271	270
query72	5696	4077	3976	3976
query73	742	324	323	323
query74	9112	8837	8858	8837
query75	3363	2625	2706	2625
query76	1381	970	970	970
query77	562	321	319	319
query78	9758	9059	9537	9059
query79	1053	547	528	528
query80	678	505	494	494
query81	462	236	232	232
query82	239	143	146	143
query83	174	149	148	148
query84	258	74	77	74
query85	676	315	412	315
query86	314	291	289	289
query87	4428	4366	4348	4348
query88	3512	2281	2279	2279
query89	371	285	277	277
query90	1753	184	183	183
query91	120	96	96	96
query92	56	50	47	47
query93	1054	534	531	531
query94	559	292	289	289
query95	315	250	251	250
query96	578	257	261	257
query97	3203	3100	3113	3100
query98	220	198	195	195
query99	1527	1282	1256	1256
Total cold run time: 301608 ms
Total hot run time: 191717 ms

doris-robot · 2024-09-03T06:10:55Z

ClickBench: Total hot run time: 31.95 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 58697ef0bf55fe752631b3303c0cb84510bbd2cd, data reload: false
query1	0.04	0.04	0.04
query2	0.07	0.03	0.04
query3	0.22	0.05	0.05
query4	1.66	0.08	0.08
query5	0.51	0.49	0.48
query6	1.13	0.72	0.73
query7	0.02	0.01	0.01
query8	0.06	0.05	0.04
query9	0.54	0.49	0.48
query10	0.54	0.53	0.54
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.61	0.59	0.58
query14	2.05	2.11	2.06
query15	0.83	0.83	0.82
query16	0.35	0.37	0.37
query17	1.01	0.96	0.99
query18	0.20	0.19	0.21
query19	1.87	1.82	1.84
query20	0.02	0.01	0.01
query21	15.40	0.66	0.65
query22	4.17	7.34	2.01
query23	18.23	1.38	1.23
query24	2.13	0.22	0.22
query25	0.16	0.08	0.07
query26	0.26	0.18	0.18
query27	0.07	0.07	0.07
query28	13.24	1.02	1.00
query29	12.67	3.27	3.28
query30	0.25	0.06	0.05
query31	2.86	0.40	0.38
query32	3.27	0.49	0.48
query33	2.96	2.95	3.02
query34	16.96	4.38	4.40
query35	4.46	4.49	4.42
query36	0.66	0.48	0.50
query37	0.19	0.16	0.16
query38	0.15	0.15	0.15
query39	0.05	0.04	0.04
query40	0.16	0.12	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.54 s
Total hot run time: 31.95 s

github-actions · 2024-09-03T15:16:27Z

Possible file(s) that should be tracked in LFS detected: 🚨

The following file(s) exceeds the file size limit: 1048576 bytes, as set in the .yml configuration files:

regression-test/data/external_table_p0/hive/test_hive_basic_type.out

Consider using git-lfs to manage large files.

github-actions · 2024-09-03T15:16:44Z

Possible file(s) that should be tracked in LFS detected: 🚨

The following file(s) exceeds the file size limit: 1048576 bytes, as set in the .yml configuration files:

regression-test/data/external_table_p0/hive/test_hive_basic_type.out

Consider using git-lfs to manage large files.

suxiaogang223 · 2024-09-03T15:19:38Z

run buildall

github-actions

clang-tidy made some suggestions

suxiaogang223 · 2024-09-04T02:33:02Z

run buildall

github-actions · 2024-09-04T02:33:27Z

Possible file(s) that should be tracked in LFS detected: 🚨

The following file(s) exceeds the file size limit: 1048576 bytes, as set in the .yml configuration files:

regression-test/data/external_table_p0/hive/test_hive_basic_type.out

Consider using git-lfs to manage large files.

doris-robot · 2024-09-04T04:17:59Z

TPC-H: Total hot run time: 38089 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0bd4324268ebd4bc89487dd837bab7ff5be52af0, data reload: false
------ Round 1 ----------------------------------
q1	17909	5215	4340	4340
q2	2022	187	179	179
q3	10870	937	1163	937
q4	10165	737	670	670
q5	7748	2881	2742	2742
q6	227	140	135	135
q7	959	616	599	599
q8	9357	2078	2098	2078
q9	7177	6560	6543	6543
q10	7017	2214	2189	2189
q11	473	242	245	242
q12	401	230	227	227
q13	17771	3146	3138	3138
q14	282	231	236	231
q15	531	497	496	496
q16	620	520	509	509
q17	994	714	716	714
q18	7427	6899	6791	6791
q19	1386	1143	1054	1054
q20	675	340	329	329
q21	4305	2965	2989	2965
q22	1143	1002	981	981
Total cold run time: 109459 ms
Total hot run time: 38089 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4364	4311	4299	4299
q2	389	276	271	271
q3	2900	2673	2662	2662
q4	1956	1648	1688	1648
q5	5640	5676	5753	5676
q6	241	139	144	139
q7	2228	1886	1872	1872
q8	3290	3426	3513	3426
q9	8890	8863	8830	8830
q10	3628	3367	3345	3345
q11	585	527	522	522
q12	826	674	698	674
q13	15963	3235	3327	3235
q14	320	295	301	295
q15	531	484	493	484
q16	605	600	565	565
q17	1845	1556	1555	1555
q18	8181	7851	7866	7851
q19	1714	1595	1403	1403
q20	2157	1884	1894	1884
q21	5721	5453	5491	5453
q22	1126	1052	1047	1047
Total cold run time: 73100 ms
Total hot run time: 57136 ms

doris-robot · 2024-09-04T04:30:08Z

TPC-DS: Total hot run time: 192975 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0bd4324268ebd4bc89487dd837bab7ff5be52af0, data reload: false
query1	1269	891	873	873
query2	6383	1934	1933	1933
query3	10628	4056	3778	3778
query4	59525	23873	23506	23506
query5	5486	498	501	498
query6	420	161	166	161
query7	5766	293	301	293
query8	295	213	218	213
query9	8739	2478	2465	2465
query10	479	281	271	271
query11	18567	15083	15176	15083
query12	162	105	113	105
query13	1544	421	405	405
query14	10818	7525	7577	7525
query15	257	178	176	176
query16	7489	503	473	473
query17	1109	564	589	564
query18	1940	315	313	313
query19	273	143	141	141
query20	122	110	107	107
query21	214	105	102	102
query22	4545	4419	4462	4419
query23	34347	33474	33460	33460
query24	5930	2866	2833	2833
query25	523	392	412	392
query26	677	157	156	156
query27	1758	281	272	272
query28	3940	2071	2052	2052
query29	651	405	399	399
query30	224	153	160	153
query31	911	768	771	768
query32	77	51	53	51
query33	485	304	288	288
query34	869	471	474	471
query35	828	731	777	731
query36	1072	917	935	917
query37	150	96	85	85
query38	4006	3887	3929	3887
query39	1426	1390	1398	1390
query40	192	118	114	114
query41	47	46	45	45
query42	113	99	96	96
query43	507	466	480	466
query44	1111	733	736	733
query45	199	164	166	164
query46	1072	717	733	717
query47	1876	1770	1786	1770
query48	368	297	301	297
query49	798	428	443	428
query50	821	423	414	414
query51	6965	6928	6970	6928
query52	101	86	87	86
query53	250	176	174	174
query54	570	462	453	453
query55	77	76	74	74
query56	280	253	251	251
query57	1222	1075	1066	1066
query58	223	231	220	220
query59	2971	2847	2811	2811
query60	303	279	271	271
query61	105	101	98	98
query62	744	665	664	664
query63	226	185	181	181
query64	2792	672	673	672
query65	3175	3149	3130	3130
query66	671	337	356	337
query67	15225	15037	15105	15037
query68	2737	558	573	558
query69	404	306	284	284
query70	1195	1135	1060	1060
query71	343	279	280	279
query72	5533	4195	4027	4027
query73	752	329	327	327
query74	9228	8821	8794	8794
query75	3346	2683	2633	2633
query76	1392	943	991	943
query77	526	342	336	336
query78	9666	9093	9021	9021
query79	1008	541	531	531
query80	697	510	522	510
query81	465	241	235	235
query82	236	141	146	141
query83	171	150	153	150
query84	260	81	85	81
query85	686	285	276	276
query86	304	272	266	266
query87	4456	4335	4346	4335
query88	2905	2302	2302	2302
query89	386	286	287	286
query90	2046	194	190	190
query91	127	100	102	100
query92	66	50	47	47
query93	1075	535	545	535
query94	701	300	295	295
query95	382	251	257	251
query96	609	260	273	260
query97	3162	3039	3118	3039
query98	221	201	203	201
query99	1520	1305	1276	1276
Total cold run time: 303513 ms
Total hot run time: 192975 ms

doris-robot · 2024-09-04T04:35:43Z

ClickBench: Total hot run time: 31.46 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0bd4324268ebd4bc89487dd837bab7ff5be52af0, data reload: false
query1	0.05	0.04	0.04
query2	0.09	0.04	0.04
query3	0.23	0.05	0.06
query4	1.64	0.10	0.10
query5	0.50	0.50	0.49
query6	1.14	0.74	0.73
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.55	0.48	0.48
query10	0.54	0.55	0.53
query11	0.16	0.12	0.12
query12	0.14	0.12	0.13
query13	0.61	0.59	0.58
query14	2.10	2.05	2.04
query15	0.84	0.81	0.82
query16	0.36	0.36	0.37
query17	1.01	1.01	1.05
query18	0.21	0.21	0.21
query19	1.90	1.73	1.75
query20	0.02	0.01	0.01
query21	15.41	0.65	0.64
query22	3.96	7.84	1.38
query23	18.26	1.39	1.33
query24	2.19	0.22	0.21
query25	0.15	0.08	0.09
query26	0.26	0.18	0.18
query27	0.07	0.07	0.08
query28	13.19	1.03	1.01
query29	12.65	3.36	3.33
query30	0.26	0.06	0.05
query31	2.88	0.41	0.40
query32	3.24	0.48	0.49
query33	3.01	3.00	3.03
query34	17.22	4.36	4.39
query35	4.43	4.37	4.45
query36	0.65	0.48	0.49
query37	0.19	0.15	0.17
query38	0.17	0.14	0.15
query39	0.04	0.03	0.04
query40	0.16	0.13	0.13
query41	0.10	0.05	0.05
query42	0.06	0.06	0.04
query43	0.05	0.04	0.04
Total cold run time: 110.76 s
Total hot run time: 31.46 s

suxiaogang223 · 2024-09-04T11:29:49Z

run buildall

github-actions · 2024-09-04T11:30:21Z

Possible file(s) that should be tracked in LFS detected: 🚨

The following file(s) exceeds the file size limit: 1048576 bytes, as set in the .yml configuration files:

regression-test/data/external_table_p0/hive/test_hive_basic_type.out

Consider using git-lfs to manage large files.

doris-robot · 2024-09-04T13:00:24Z

TPC-H: Total hot run time: 38304 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3ba611c450405e55f6bba4e08f5999a4c6862614, data reload: false
------ Round 1 ----------------------------------
q1	17828	4637	4484	4484
q2	2880	179	180	179
q3	10779	1121	1144	1121
q4	10542	750	797	750
q5	8037	2939	2889	2889
q6	236	142	141	141
q7	986	628	607	607
q8	9505	2098	2102	2098
q9	6978	6604	6591	6591
q10	7014	2242	2230	2230
q11	456	234	242	234
q12	401	222	218	218
q13	17781	3090	3111	3090
q14	285	241	242	241
q15	540	504	480	480
q16	585	497	526	497
q17	995	629	776	629
q18	7419	7046	6761	6761
q19	1388	1005	1153	1005
q20	680	326	321	321
q21	3960	3036	2727	2727
q22	1135	1011	1019	1011
Total cold run time: 110410 ms
Total hot run time: 38304 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4387	4359	4293	4293
q2	369	265	282	265
q3	2913	2718	2637	2637
q4	1899	1659	1690	1659
q5	5427	5407	5456	5407
q6	225	128	134	128
q7	2119	1738	1783	1738
q8	3219	3368	3326	3326
q9	8487	8458	8527	8458
q10	3497	3234	3193	3193
q11	612	503	504	503
q12	797	638	599	599
q13	11555	3102	3034	3034
q14	302	269	272	269
q15	523	468	491	468
q16	600	579	553	553
q17	1816	1515	1484	1484
q18	7743	7490	7503	7490
q19	1673	1449	1490	1449
q20	2064	1864	1849	1849
q21	5330	5116	5183	5116
q22	1099	1042	1030	1030
Total cold run time: 66656 ms
Total hot run time: 54948 ms

doris-robot · 2024-09-04T13:12:04Z

TPC-DS: Total hot run time: 187944 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3ba611c450405e55f6bba4e08f5999a4c6862614, data reload: false
query1	923	373	368	368
query2	6499	2021	1934	1934
query3	6644	205	210	205
query4	34212	23307	23161	23161
query5	4158	484	482	482
query6	259	171	155	155
query7	4580	288	289	288
query8	247	200	210	200
query9	8596	2462	2440	2440
query10	442	261	247	247
query11	16603	15073	15068	15068
query12	155	96	98	96
query13	1633	378	364	364
query14	8746	7206	6957	6957
query15	294	199	166	166
query16	8041	441	413	413
query17	1588	572	538	538
query18	2058	291	281	281
query19	221	144	139	139
query20	119	127	108	108
query21	211	102	101	101
query22	4581	4180	4219	4180
query23	34078	33341	33344	33341
query24	11126	2913	2809	2809
query25	633	387	394	387
query26	1187	156	152	152
query27	2553	282	270	270
query28	7353	2068	2046	2046
query29	846	414	404	404
query30	310	159	151	151
query31	973	763	784	763
query32	95	54	58	54
query33	749	288	280	280
query34	1002	469	481	469
query35	836	760	693	693
query36	1093	937	902	902
query37	168	101	91	91
query38	4106	3946	3961	3946
query39	1431	1401	1425	1401
query40	195	116	115	115
query41	49	43	44	43
query42	118	92	94	92
query43	524	476	488	476
query44	1182	745	752	745
query45	198	167	169	167
query46	1106	734	738	734
query47	1938	1843	1789	1789
query48	371	295	293	293
query49	1088	429	458	429
query50	793	408	416	408
query51	7041	6887	6932	6887
query52	96	88	87	87
query53	253	180	178	178
query54	976	450	449	449
query55	77	77	73	73
query56	277	260	261	260
query57	1228	1090	1111	1090
query58	233	238	256	238
query59	3013	2829	2918	2829
query60	306	265	263	263
query61	103	103	99	99
query62	861	669	658	658
query63	225	184	178	178
query64	4222	679	692	679
query65	3184	3154	3132	3132
query66	1418	339	333	333
query67	15690	15495	15486	15486
query68	3281	582	575	575
query69	391	275	280	275
query70	1179	1129	997	997
query71	330	269	271	269
query72	6418	4100	3965	3965
query73	739	323	328	323
query74	9098	9109	8911	8911
query75	3411	2702	2657	2657
query76	1937	964	991	964
query77	508	310	319	310
query78	9917	9032	9081	9032
query79	1029	533	531	531
query80	682	523	551	523
query81	463	248	238	238
query82	240	143	152	143
query83	175	158	161	158
query84	242	85	79	79
query85	751	361	341	341
query86	324	295	296	295
query87	4409	4329	4282	4282
query88	3167	2448	2316	2316
query89	387	289	281	281
query90	1903	192	190	190
query91	137	116	113	113
query92	63	53	50	50
query93	1036	548	529	529
query94	719	291	300	291
query95	357	263	258	258
query96	582	267	261	261
query97	3204	3102	3076	3076
query98	217	198	212	198
query99	1505	1296	1253	1253
Total cold run time: 284915 ms
Total hot run time: 187944 ms

doris-robot · 2024-09-12T05:42:40Z

ClickBench: Total hot run time: 30.84 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2bcc273a153417ee1e079a263b829a7e3924bd42, data reload: false
query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.23	0.04	0.05
query4	1.67	0.08	0.07
query5	0.51	0.50	0.49
query6	1.14	0.72	0.73
query7	0.02	0.01	0.02
query8	0.06	0.04	0.04
query9	0.57	0.53	0.53
query10	0.56	0.58	0.57
query11	0.16	0.12	0.12
query12	0.16	0.12	0.12
query13	0.64	0.61	0.61
query14	1.44	1.47	1.45
query15	0.93	0.89	0.88
query16	0.36	0.36	0.37
query17	1.03	1.05	1.02
query18	0.16	0.16	0.16
query19	1.94	1.84	1.85
query20	0.01	0.01	0.01
query21	15.43	0.70	0.67
query22	4.14	7.60	1.22
query23	17.78	1.38	1.30
query24	2.29	0.22	0.21
query25	0.19	0.08	0.09
query26	0.29	0.18	0.18
query27	0.07	0.08	0.08
query28	13.16	1.01	0.97
query29	12.53	3.28	3.34
query30	0.25	0.05	0.06
query31	2.88	0.44	0.42
query32	3.23	0.49	0.49
query33	3.05	3.05	3.02
query34	15.43	4.31	4.34
query35	4.33	4.34	4.34
query36	0.68	0.50	0.49
query37	0.19	0.17	0.16
query38	0.16	0.14	0.15
query39	0.04	0.04	0.04
query40	0.16	0.13	0.14
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.05	0.04
Total cold run time: 108.19 s
Total hot run time: 30.84 s

suxiaogang223 · 2024-09-12T09:46:10Z

run external

suxiaogang223 · 2024-09-12T13:10:06Z

run buildall

github-actions · 2024-09-12T13:10:14Z

Possible file(s) that should be tracked in LFS detected: 🚨

The following file(s) exceeds the file size limit: 1048576 bytes, as set in the .yml configuration files:

regression-test/data/external_table_p0/hive/test_hive_basic_type.out

Consider using git-lfs to manage large files.

doris-robot · 2024-09-12T14:48:14Z

TPC-H: Total hot run time: 43229 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 37882bc89142713faaedf8223dfa85fb7598c89b, data reload: false
------ Round 1 ----------------------------------
q1	17639	7470	7292	7292
q2	2042	193	192	192
q3	10535	1464	1419	1419
q4	10243	1024	1047	1024
q5	7718	3232	3169	3169
q6	242	155	155	155
q7	1050	649	632	632
q8	9464	2018	2008	2008
q9	6862	6297	6343	6297
q10	7049	2552	2567	2552
q11	431	261	262	261
q12	417	226	232	226
q13	17761	3044	3053	3044
q14	290	257	251	251
q15	570	540	531	531
q16	521	447	424	424
q17	1017	961	963	961
q18	7548	6684	6766	6684
q19	1377	1248	1249	1248
q20	614	346	329	329
q21	3889	3549	3543	3543
q22	1094	987	1015	987
Total cold run time: 108373 ms
Total hot run time: 43229 ms
----- Round 2, with runtime_filter_mode=off -----
q1	7198	7204	7238	7204
q2	343	240	232	232
q3	2936	2908	2911	2908
q4	1995	1994	1981	1981
q5	5437	5425	5462	5425
q6	241	148	145	145
q7	2069	1658	1642	1642
q8	3254	3333	3328	3328
q9	8449	8434	8418	8418
q10	3422	3459	3468	3459
q11	589	482	473	473
q12	776	595	554	554
q13	8669	3081	3036	3036
q14	302	269	281	269
q15	578	518	514	514
q16	489	443	447	443
q17	1778	1721	1724	1721
q18	8086	7596	7458	7458
q19	1742	1712	1708	1708
q20	2061	1818	1799	1799
q21	5660	5399	5446	5399
q22	1139	987	993	987
Total cold run time: 67213 ms
Total hot run time: 59103 ms

doris-robot · 2024-09-12T14:54:51Z

TeamCity be ut coverage result:
Function Coverage: 36.89% (9464/25652)
Line Coverage: 28.27% (77852/275433)
Region Coverage: 27.67% (40204/145304)
Branch Coverage: 24.27% (20427/84158)
Coverage Report: http://coverage.selectdb-in.cc/coverage/37882bc89142713faaedf8223dfa85fb7598c89b_37882bc89142713faaedf8223dfa85fb7598c89b/report/index.html

doris-robot · 2024-09-12T14:59:55Z

TPC-DS: Total hot run time: 195210 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 37882bc89142713faaedf8223dfa85fb7598c89b, data reload: false
query1	935	381	373	373
query2	6473	1807	1812	1807
query3	6660	215	222	215
query4	25890	23985	24060	23985
query5	4988	558	528	528
query6	270	171	163	163
query7	4593	303	291	291
query8	287	210	210	210
query9	8524	2616	2609	2609
query10	469	279	286	279
query11	16128	15658	15611	15611
query12	167	104	109	104
query13	1677	404	381	381
query14	11508	6965	7032	6965
query15	220	178	168	168
query16	7579	484	468	468
query17	1558	592	584	584
query18	1985	299	302	299
query19	198	153	152	152
query20	124	113	114	113
query21	212	107	107	107
query22	4582	4251	4072	4072
query23	34803	33912	33890	33890
query24	9153	3199	3137	3137
query25	631	381	379	379
query26	719	157	158	157
query27	2133	277	285	277
query28	5219	2113	2096	2096
query29	891	410	415	410
query30	294	156	154	154
query31	991	768	782	768
query32	102	54	58	54
query33	632	288	307	288
query34	914	483	497	483
query35	903	765	751	751
query36	1079	882	898	882
query37	136	80	84	80
query38	4002	3863	3903	3863
query39	1467	1397	1412	1397
query40	208	117	115	115
query41	46	49	46	46
query42	117	100	100	100
query43	485	437	430	430
query44	1197	804	770	770
query45	198	171	174	171
query46	1106	846	814	814
query47	1905	1755	1835	1755
query48	352	283	285	283
query49	1066	449	450	449
query50	928	438	439	438
query51	7206	6940	7028	6940
query52	98	89	92	89
query53	258	185	187	185
query54	791	458	469	458
query55	78	77	77	77
query56	294	271	263	263
query57	1181	1078	1088	1078
query58	248	235	235	235
query59	2752	2658	2497	2497
query60	310	275	276	275
query61	104	103	102	102
query62	927	682	664	664
query63	224	192	190	190
query64	3683	686	694	686
query65	3264	3210	3183	3183
query66	818	300	305	300
query67	16092	15529	15506	15506
query68	3161	869	858	858
query69	451	323	338	323
query70	1195	1163	1182	1163
query71	357	343	342	342
query72	5855	3343	3408	3343
query73	589	582	572	572
query74	9261	9054	9046	9046
query75	3169	3044	3019	3019
query76	1903	840	854	840
query77	502	413	418	413
query78	9476	9181	9314	9181
query79	918	904	879	879
query80	840	839	814	814
query81	457	267	268	267
query82	265	266	266	266
query83	193	194	193	193
query84	233	113	110	110
query85	665	471	403	403
query86	325	311	326	311
query87	4372	4385	4464	4385
query88	4058	4010	3995	3995
query89	380	366	372	366
query90	1298	318	315	315
query91	129	125	123	123
query92	80	78	77	77
query93	1027	1040	1023	1023
query94	622	369	371	369
query95	463	436	428	428
query96	463	472	464	464
query97	3135	3139	3143	3139
query98	237	228	227	227
query99	1666	1307	1307	1307
Total cold run time: 272777 ms
Total hot run time: 195210 ms

doris-robot · 2024-09-12T15:05:27Z

ClickBench: Total hot run time: 31.19 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 37882bc89142713faaedf8223dfa85fb7598c89b, data reload: false
query1	0.05	0.04	0.04
query2	0.07	0.04	0.04
query3	0.22	0.04	0.04
query4	1.68	0.07	0.07
query5	0.49	0.51	0.51
query6	1.14	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.05	0.05
query9	0.56	0.53	0.51
query10	0.59	0.59	0.58
query11	0.16	0.12	0.12
query12	0.16	0.12	0.12
query13	0.63	0.62	0.61
query14	1.44	1.50	1.44
query15	0.92	0.86	0.88
query16	0.36	0.36	0.36
query17	1.02	1.03	1.04
query18	0.17	0.16	0.17
query19	1.95	1.88	1.83
query20	0.02	0.01	0.00
query21	15.40	0.67	0.67
query22	3.89	7.50	1.40
query23	18.08	1.28	1.42
query24	2.25	0.23	0.22
query25	0.19	0.08	0.09
query26	0.28	0.18	0.18
query27	0.08	0.08	0.07
query28	13.18	1.14	1.10
query29	12.53	3.38	3.34
query30	0.24	0.05	0.05
query31	2.86	0.42	0.42
query32	3.23	0.51	0.49
query33	3.06	3.07	3.06
query34	15.45	4.33	4.30
query35	4.35	4.34	4.37
query36	0.70	0.48	0.51
query37	0.18	0.16	0.15
query38	0.16	0.14	0.15
query39	0.04	0.04	0.03
query40	0.17	0.15	0.14
query41	0.10	0.04	0.04
query42	0.06	0.06	0.05
query43	0.06	0.05	0.05
Total cold run time: 108.24 s
Total hot run time: 31.19 s

morningman

LGTM

github-actions · 2024-09-18T06:34:36Z

PR approved by at least one committer and no changes requested.

kaka11chen

LGTM

…#40291) ## Proposed changes related pr: apache#39700 support escape.delim and serialization.null.format for hive text

## Proposed changes pick from master: #40315 #40291 #40183 #38549

…#40291) ## Proposed changes related pr: apache#39700 support escape.delim and serialization.null.format for hive text

…format for hive text (#41684) ## Proposed changes pick from master: #40291

…#40291) ## Proposed changes related pr: apache#39700 support escape.delim and serialization.null.format for hive text

github-actionsBot added the lfs-detected! Warning Label for use when LFS is detected in the commits of a Pull Request label Sep 2, 2024

suxiaogang223 force-pushed the hive_text_null_format branch from 338d90f to b2e4cedCompare September 3, 2024 15:16

github-actionsBot reviewed Sep 3, 2024
View reviewed changes

Comment threadbe/src/vec/exec/format/csv/csv_reader.cpp

fix regress
37882bc

morningman added dev/2.1.x-experimental and removed dev/2.1.x labels Sep 13, 2024

morningman approved these changes Sep 18, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 18, 2024

kaka11chen approved these changes Sep 19, 2024
View reviewed changes

morningman merged commit ba6102b into apache:masterSep 27, 2024

suxiaogang223 deleted the hive_text_null_format branch September 27, 2024 04:49

suxiaogang223 mentioned this pull request Oct 10, 2024
[charry-pick](branch3.0) support text hive #41656
Merged

morningman pushed a commit that referenced this pull request Oct 10, 2024
[charry-pick](branch3.0) support text hive (#41656)
eb419c2
## Proposed changes pick from master: #40315 #40291 #40183 #38549

morningman added dev/3.0.3-merged and removed dev/3.0.x labels Oct 10, 2024

suxiaogang223 mentioned this pull request Oct 11, 2024
[cherry-pick](branch2.1) support escape.delim and serialization.null.format for hive text #41684
Merged

cjj2010 pushed a commit to cjj2010/doris that referenced this pull request Oct 12, 2024
[fix](hive) support null_format and escape_char for hive text (apache…
22d8e71
…#40291) ## Proposed changes related pr: apache#39700 support escape.delim and serialization.null.format for hive text

yiguolei pushed a commit that referenced this pull request Oct 14, 2024
[cherry-pick](branch2.1) support escape.delim and serialization.null.…
4888c63
…format for hive text (#41684) ## Proposed changes pick from master: #40291

yiguolei added dev/2.1.7-merged and removed dev/2.1.x-experimental labels Oct 14, 2024

Uh oh!

Conversation

suxiaogang223 commented Sep 2, 2024

Proposed changes

Uh oh!

doris-robot commented Sep 2, 2024

Uh oh!

github-actionsBot commented Sep 2, 2024

Possible file(s) that should be tracked in LFS detected: 🚨

Uh oh!

github-actionsBot commented Sep 2, 2024

Uh oh!

github-actionsBot commented Sep 2, 2024

Possible file(s) that should be tracked in LFS detected: 🚨

Uh oh!

suxiaogang223 commented Sep 2, 2024

Uh oh!

github-actionsBot commented Sep 2, 2024

Uh oh!

doris-robot commented Sep 2, 2024

Uh oh!

doris-robot commented Sep 2, 2024

Uh oh!

doris-robot commented Sep 2, 2024

Uh oh!

github-actionsBot commented Sep 3, 2024

Possible file(s) that should be tracked in LFS detected: 🚨

Uh oh!

suxiaogang223 commented Sep 3, 2024

Uh oh!

github-actionsBot commented Sep 3, 2024

Uh oh!

doris-robot commented Sep 3, 2024

Uh oh!

doris-robot commented Sep 3, 2024

Uh oh!

doris-robot commented Sep 3, 2024

Uh oh!

github-actionsBot commented Sep 3, 2024

Possible file(s) that should be tracked in LFS detected: 🚨

Uh oh!

github-actionsBot commented Sep 3, 2024

Possible file(s) that should be tracked in LFS detected: 🚨

Uh oh!

suxiaogang223 commented Sep 3, 2024

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

suxiaogang223 commented Sep 4, 2024

Uh oh!

github-actionsBot commented Sep 4, 2024

Possible file(s) that should be tracked in LFS detected: 🚨

Uh oh!

doris-robot commented Sep 4, 2024

Uh oh!

doris-robot commented Sep 4, 2024

Uh oh!

doris-robot commented Sep 4, 2024

Uh oh!

suxiaogang223 commented Sep 4, 2024

Uh oh!

github-actionsBot commented Sep 4, 2024

Possible file(s) that should be tracked in LFS detected: 🚨

Uh oh!

doris-robot commented Sep 4, 2024

Uh oh!

doris-robot commented Sep 4, 2024

Uh oh!

doris-robot commented Sep 12, 2024

Uh oh!

suxiaogang223 commented Sep 12, 2024

Uh oh!

suxiaogang223 commented Sep 12, 2024

Uh oh!

github-actionsBot commented Sep 12, 2024

Possible file(s) that should be tracked in LFS detected: 🚨

Uh oh!

doris-robot commented Sep 12, 2024