Skip to content

[fix](stream loader) do not redirect to https for stream load - #50009

Merged
englefly merged 3 commits into
apache:masterfrom
englefly:auditStreamloaderHttps
May 7, 2025
Merged

[fix](stream loader) do not redirect to https for stream load#50009
englefly merged 3 commits into
apache:masterfrom
englefly:auditStreamloaderHttps

Conversation

@englefly

@engleflyenglefly commented Apr 14, 2025

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

auditlog stream load plugin does not work when https is enabled.
The reason is when https is enabled, LoadAction will add an extra redirection(from http to https), which breaks the process of audit plugin.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen

hello-stephen commented Apr 14, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@englefly

englefly commented Apr 14, 2025

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Apr 14, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 34183 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c3c8e4add265c732b9233fa22e72731c59af2dce, data reload: false
------ Round 1 ----------------------------------
q1	25909	5105	5052	5052
q2	2065	266	177	177
q3	10407	1223	678	678
q4	10245	975	531	531
q5	7504	2351	2366	2351
q6	186	163	137	137
q7	911	742	620	620
q8	9326	1318	1116	1116
q9	6845	5076	5064	5064
q10	6807	2309	1868	1868
q11	482	283	266	266
q12	342	350	217	217
q13	17793	3697	3044	3044
q14	229	229	201	201
q15	528	486	500	486
q16	631	630	604	604
q17	571	848	357	357
q18	7494	7201	7307	7201
q19	1209	941	579	579
q20	339	339	224	224
q21	4360	3367	2449	2449
q22	1058	1037	961	961
Total cold run time: 115241 ms
Total hot run time: 34183 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5104	5162	5378	5162
q2	237	326	230	230
q3	2180	2652	2355	2355
q4	1470	1865	1534	1534
q5	4565	4531	4519	4519
q6	225	172	131	131
q7	2075	2015	1826	1826
q8	2623	2637	2563	2563
q9	7410	7244	7305	7244
q10	3024	3221	2773	2773
q11	582	507	498	498
q12	692	760	609	609
q13	3578	3838	3431	3431
q14	277	300	270	270
q15	518	478	478	478
q16	664	678	655	655
q17	1153	1559	1401	1401
q18	7713	7567	7428	7428
q19	849	897	994	897
q20	1913	1980	1838	1838
q21	5336	4841	4813	4813
q22	1084	1060	1018	1018
Total cold run time: 53272 ms
Total hot run time: 51673 ms

@doris-robot

doris-robot commented Apr 14, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 192747 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c3c8e4add265c732b9233fa22e72731c59af2dce, data reload: false
query1	1397	1058	1044	1044
query2	6223	1957	1913	1913
query3	10995	4489	4367	4367
query4	25597	23760	23224	23224
query5	4205	669	460	460
query6	311	209	207	207
query7	3987	507	283	283
query8	314	281	245	245
query9	8490	2594	2602	2594
query10	456	338	267	267
query11	15332	15049	14738	14738
query12	162	119	112	112
query13	1553	539	391	391
query14	8917	6245	6250	6245
query15	220	186	167	167
query16	7216	638	462	462
query17	1176	736	577	577
query18	1998	412	297	297
query19	197	192	179	179
query20	127	113	123	113
query21	205	123	110	110
query22	4464	4505	4264	4264
query23	34366	33608	33447	33447
query24	8456	2509	2421	2421
query25	538	465	429	429
query26	1155	265	151	151
query27	2725	522	344	344
query28	4600	2469	2422	2422
query29	750	577	463	463
query30	275	226	198	198
query31	929	895	802	802
query32	79	71	65	65
query33	555	386	340	340
query34	833	903	566	566
query35	834	880	747	747
query36	962	991	913	913
query37	126	107	85	85
query38	4229	4244	4201	4201
query39	1512	1463	1437	1437
query40	224	128	118	118
query41	64	56	58	56
query42	124	159	111	111
query43	529	520	486	486
query44	1417	830	829	829
query45	180	176	177	176
query46	878	1068	672	672
query47	1815	1845	1831	1831
query48	396	437	315	315
query49	752	510	433	433
query50	696	721	415	415
query51	4315	4321	4222	4222
query52	116	109	107	107
query53	243	270	216	216
query54	600	581	516	516
query55	79	84	83	83
query56	319	302	295	295
query57	1176	1214	1111	1111
query58	272	269	259	259
query59	2588	2795	2599	2599
query60	358	342	310	310
query61	135	124	130	124
query62	802	736	678	678
query63	241	198	194	194
query64	4145	1065	719	719
query65	4535	4337	4457	4337
query66	1088	425	323	323
query67	15958	15485	15355	15355
query68	8393	892	512	512
query69	489	313	280	280
query70	1191	1082	1085	1082
query71	449	327	299	299
query72	5607	4845	4836	4836
query73	703	654	354	354
query74	9106	9147	8716	8716
query75	3853	3189	2694	2694
query76	3659	1198	768	768
query77	774	391	269	269
query78	10041	10164	9272	9272
query79	2661	830	575	575
query80	624	511	449	449
query81	487	258	225	225
query82	463	125	98	98
query83	292	251	244	244
query84	290	105	84	84
query85	779	388	315	315
query86	339	305	275	275
query87	4500	4424	4272	4272
query88	3293	2233	2230	2230
query89	415	319	290	290
query90	1854	213	221	213
query91	141	138	110	110
query92	69	62	57	57
query93	1537	958	580	580
query94	667	426	298	298
query95	376	289	296	289
query96	483	563	274	274
query97	3142	3211	3144	3144
query98	226	210	205	205
query99	1460	1386	1258	1258
Total cold run time: 278874 ms
Total hot run time: 192747 ms

@doris-robot

doris-robot commented Apr 14, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.27 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c3c8e4add265c732b9233fa22e72731c59af2dce, data reload: false
query1	0.04	0.04	0.03
query2	0.12	0.11	0.11
query3	0.25	0.20	0.19
query4	1.60	0.19	0.19
query5	0.59	0.57	0.58
query6	1.17	0.71	0.73
query7	0.02	0.01	0.02
query8	0.04	0.03	0.04
query9	0.58	0.53	0.52
query10	0.56	0.57	0.56
query11	0.16	0.11	0.10
query12	0.14	0.12	0.11
query13	0.62	0.60	0.60
query14	2.67	2.81	2.68
query15	0.94	0.85	0.85
query16	0.37	0.38	0.38
query17	1.00	1.02	1.03
query18	0.21	0.20	0.19
query19	1.85	1.87	1.92
query20	0.02	0.01	0.01
query21	15.36	0.90	0.56
query22	0.77	1.22	0.66
query23	14.85	1.42	0.61
query24	6.85	1.93	0.97
query25	0.49	0.19	0.12
query26	0.67	0.16	0.13
query27	0.04	0.05	0.05
query28	9.20	0.87	0.43
query29	12.55	4.03	3.30
query30	0.25	0.09	0.07
query31	2.82	0.62	0.39
query32	3.22	0.56	0.47
query33	3.11	3.09	3.18
query34	15.78	5.11	4.43
query35	4.52	4.52	4.46
query36	0.68	0.52	0.50
query37	0.09	0.07	0.06
query38	0.04	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.12	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 104.59 s
Total hot run time: 31.27 s

@englefly

englefly commented Apr 15, 2025

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Apr 15, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 34057 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c3c8e4add265c732b9233fa22e72731c59af2dce, data reload: false
------ Round 1 ----------------------------------
q1	26312	5031	5026	5026
q2	2079	283	184	184
q3	10395	1266	692	692
q4	10222	1007	517	517
q5	7836	2396	2358	2358
q6	179	164	132	132
q7	907	760	605	605
q8	9322	1260	1103	1103
q9	6743	5119	5082	5082
q10	6842	2313	1884	1884
q11	488	277	264	264
q12	357	351	223	223
q13	17771	3638	3108	3108
q14	233	229	203	203
q15	534	475	493	475
q16	622	607	577	577
q17	579	839	345	345
q18	7423	7200	7110	7110
q19	1384	939	555	555
q20	324	338	223	223
q21	3848	3458	2407	2407
q22	1067	1053	984	984
Total cold run time: 115467 ms
Total hot run time: 34057 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5162	5149	5047	5047
q2	245	339	230	230
q3	2248	2728	2380	2380
q4	1442	1862	1459	1459
q5	4576	4496	4484	4484
q6	220	169	130	130
q7	1993	1921	1771	1771
q8	2635	2550	2551	2550
q9	7323	7233	7053	7053
q10	3039	3227	2764	2764
q11	590	504	491	491
q12	731	771	600	600
q13	3487	3901	3318	3318
q14	306	306	276	276
q15	509	488	481	481
q16	636	683	668	668
q17	1147	1531	1418	1418
q18	8307	8092	8098	8092
q19	1176	838	921	838
q20	1976	1997	1834	1834
q21	5447	4847	4808	4808
q22	1090	1085	1044	1044
Total cold run time: 54285 ms
Total hot run time: 51736 ms

@doris-robot

doris-robot commented Apr 15, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 192622 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c3c8e4add265c732b9233fa22e72731c59af2dce, data reload: false
query1	1386	1091	1090	1090
query2	6328	1850	1870	1850
query3	10995	4523	4418	4418
query4	53642	25976	22960	22960
query5	4936	532	452	452
query6	319	204	200	200
query7	4879	511	283	283
query8	315	252	246	246
query9	5593	2607	2573	2573
query10	432	327	277	277
query11	15002	15054	14865	14865
query12	157	112	106	106
query13	1050	492	397	397
query14	9996	6418	6333	6333
query15	204	191	174	174
query16	7115	664	522	522
query17	1083	763	591	591
query18	1562	420	320	320
query19	206	203	175	175
query20	130	128	124	124
query21	210	124	108	108
query22	4380	4509	4296	4296
query23	34239	33598	33426	33426
query24	6495	2454	2411	2411
query25	456	473	404	404
query26	710	283	159	159
query27	2220	503	337	337
query28	3329	2433	2433	2433
query29	580	569	440	440
query30	280	239	219	219
query31	884	899	791	791
query32	72	61	59	59
query33	453	354	308	308
query34	762	877	526	526
query35	784	875	816	816
query36	941	1006	854	854
query37	115	100	73	73
query38	4149	4280	4284	4280
query39	1483	1424	1428	1424
query40	224	123	107	107
query41	54	56	50	50
query42	121	113	109	109
query43	507	504	477	477
query44	1303	822	819	819
query45	187	173	166	166
query46	856	1028	665	665
query47	1873	1829	1808	1808
query48	389	422	318	318
query49	671	507	432	432
query50	655	704	419	419
query51	4268	4395	4255	4255
query52	108	112	100	100
query53	229	254	185	185
query54	573	582	514	514
query55	93	82	81	81
query56	315	339	291	291
query57	1189	1201	1126	1126
query58	270	259	263	259
query59	2654	2811	2657	2657
query60	325	318	307	307
query61	126	128	127	127
query62	733	728	672	672
query63	223	191	189	189
query64	2152	1059	682	682
query65	4316	4239	4231	4231
query66	711	402	305	305
query67	15752	15608	15385	15385
query68	7435	822	511	511
query69	536	298	259	259
query70	1221	1131	1115	1115
query71	506	325	299	299
query72	5958	4812	4961	4812
query73	1476	685	350	350
query74	9224	9123	8681	8681
query75	3865	3175	2708	2708
query76	4269	1203	737	737
query77	670	377	295	295
query78	10117	10509	9291	9291
query79	2329	801	574	574
query80	618	497	449	449
query81	493	253	225	225
query82	456	126	97	97
query83	338	263	234	234
query84	284	101	94	94
query85	784	364	313	313
query86	372	294	299	294
query87	4607	4521	4319	4319
query88	3337	2207	2195	2195
query89	397	321	304	304
query90	1953	204	204	204
query91	140	143	115	115
query92	82	62	55	55
query93	1632	927	581	581
query94	679	404	300	300
query95	370	299	278	278
query96	474	568	270	270
query97	3169	3225	3142	3142
query98	230	214	206	206
query99	1412	1401	1259	1259
Total cold run time: 298182 ms
Total hot run time: 192622 ms

@doris-robot

doris-robot commented Apr 15, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.76 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c3c8e4add265c732b9233fa22e72731c59af2dce, data reload: false
query1	0.04	0.04	0.03
query2	0.13	0.10	0.12
query3	0.25	0.19	0.19
query4	1.59	0.20	0.19
query5	0.63	0.57	0.58
query6	1.19	0.71	0.72
query7	0.02	0.02	0.01
query8	0.04	0.04	0.03
query9	0.57	0.54	0.53
query10	0.57	0.57	0.58
query11	0.15	0.10	0.11
query12	0.15	0.12	0.12
query13	0.61	0.60	0.60
query14	2.80	2.70	2.71
query15	0.90	0.84	0.87
query16	0.38	0.37	0.39
query17	1.00	1.01	1.00
query18	0.21	0.19	0.19
query19	1.86	1.93	1.81
query20	0.02	0.01	0.01
query21	15.38	0.87	0.54
query22	0.77	1.33	0.71
query23	14.72	1.38	0.61
query24	7.70	1.04	0.30
query25	0.42	0.14	0.15
query26	0.67	0.17	0.13
query27	0.05	0.04	0.04
query28	8.86	0.88	0.44
query29	12.53	3.97	3.38
query30	0.24	0.09	0.06
query31	2.82	0.60	0.38
query32	3.22	0.53	0.46
query33	3.03	3.13	3.14
query34	15.76	5.03	4.48
query35	4.54	4.52	4.49
query36	0.66	0.51	0.49
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.02	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.96 s
Total hot run time: 30.76 s

@englefly
englefly marked this pull request as ready for review April 22, 2025 00:23
@englefly

englefly commented Apr 22, 2025

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Apr 22, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 33892 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f9f7f9ea2da16cae270199efa2c21b998e1b4529, data reload: false
------ Round 1 ----------------------------------
q1	25647	5036	5041	5036
q2	2063	300	182	182
q3	10366	1231	687	687
q4	10215	995	518	518
q5	7530	2397	2285	2285
q6	181	162	131	131
q7	916	766	608	608
q8	9325	1196	1081	1081
q9	6879	5143	5094	5094
q10	6799	2297	1871	1871
q11	469	278	270	270
q12	351	366	223	223
q13	17749	3683	3107	3107
q14	217	222	208	208
q15	530	480	486	480
q16	449	451	420	420
q17	600	855	371	371
q18	7528	7298	7131	7131
q19	1807	961	581	581
q20	332	330	220	220
q21	3960	2636	2421	2421
q22	1044	1016	967	967
Total cold run time: 114957 ms
Total hot run time: 33892 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5173	5046	5061	5046
q2	239	327	230	230
q3	2132	2606	2274	2274
q4	1408	1841	1408	1408
q5	4382	4326	4429	4326
q6	219	169	127	127
q7	2001	1890	1755	1755
q8	2596	2477	2475	2475
q9	7195	7212	6899	6899
q10	3033	3205	2731	2731
q11	579	509	477	477
q12	705	744	594	594
q13	3543	3853	3274	3274
q14	280	293	276	276
q15	538	518	510	510
q16	500	533	454	454
q17	1130	1513	1353	1353
q18	7629	7706	7315	7315
q19	816	896	1103	896
q20	1970	2077	1803	1803
q21	5137	4899	4873	4873
q22	1118	1074	1049	1049
Total cold run time: 52323 ms
Total hot run time: 50145 ms

@doris-robot

doris-robot commented Apr 22, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 192707 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f9f7f9ea2da16cae270199efa2c21b998e1b4529, data reload: false
query1	1404	1098	1041	1041
query2	6386	1762	1745	1745
query3	11178	4721	4619	4619
query4	53304	24676	23025	23025
query5	5185	520	470	470
query6	349	207	195	195
query7	4927	505	283	283
query8	318	236	231	231
query9	5692	2573	2587	2573
query10	413	312	247	247
query11	15094	15003	14793	14793
query12	155	109	103	103
query13	1023	499	398	398
query14	10031	6409	6385	6385
query15	218	194	174	174
query16	7078	655	526	526
query17	1122	722	572	572
query18	1582	417	316	316
query19	192	188	159	159
query20	128	119	119	119
query21	212	123	106	106
query22	4444	4396	4318	4318
query23	34005	33261	33446	33261
query24	6486	2434	2467	2434
query25	492	459	402	402
query26	678	276	154	154
query27	2181	512	341	341
query28	3100	2182	2151	2151
query29	591	571	418	418
query30	287	218	190	190
query31	852	861	807	807
query32	79	66	64	64
query33	469	366	319	319
query34	760	875	521	521
query35	803	847	780	780
query36	940	1002	914	914
query37	118	99	80	80
query38	4262	4196	4099	4099
query39	1493	1445	1442	1442
query40	206	124	104	104
query41	83	51	50	50
query42	124	112	109	109
query43	490	504	478	478
query44	1333	817	797	797
query45	180	182	166	166
query46	829	1027	672	672
query47	1878	1884	1799	1799
query48	383	418	303	303
query49	701	520	421	421
query50	678	719	412	412
query51	4266	4247	4220	4220
query52	105	104	95	95
query53	237	260	184	184
query54	592	606	530	530
query55	85	82	81	81
query56	322	301	316	301
query57	1155	1180	1168	1168
query58	259	249	273	249
query59	2676	2746	2630	2630
query60	326	323	296	296
query61	142	130	126	126
query62	766	769	689	689
query63	237	191	188	188
query64	1541	1145	825	825
query65	4435	4271	4230	4230
query66	761	419	321	321
query67	15897	15504	15328	15328
query68	7077	884	529	529
query69	543	305	313	305
query70	1212	1150	1124	1124
query71	452	313	284	284
query72	5982	4773	4843	4773
query73	1252	672	351	351
query74	8924	9088	9058	9058
query75	3285	3167	2814	2814
query76	3796	1183	764	764
query77	556	359	286	286
query78	9930	10138	9295	9295
query79	2485	826	580	580
query80	642	515	465	465
query81	518	252	220	220
query82	430	127	94	94
query83	334	260	238	238
query84	306	109	83	83
query85	787	352	317	317
query86	371	322	275	275
query87	4390	4478	4299	4299
query88	3432	2208	2214	2208
query89	401	305	286	286
query90	1830	216	221	216
query91	139	143	113	113
query92	73	59	57	57
query93	1603	948	589	589
query94	697	422	324	324
query95	368	289	291	289
query96	482	569	273	273
query97	3159	3195	3161	3161
query98	229	212	197	197
query99	1458	1402	1293	1293
Total cold run time: 295861 ms
Total hot run time: 192707 ms

@doris-robot

doris-robot commented Apr 22, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 29.67 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f9f7f9ea2da16cae270199efa2c21b998e1b4529, data reload: false
query1	0.03	0.04	0.03
query2	0.12	0.10	0.12
query3	0.25	0.19	0.19
query4	1.59	0.20	0.19
query5	0.61	0.57	0.58
query6	1.20	0.72	0.71
query7	0.02	0.02	0.02
query8	0.04	0.04	0.04
query9	0.58	0.52	0.52
query10	0.57	0.56	0.55
query11	0.15	0.10	0.11
query12	0.15	0.11	0.11
query13	0.62	0.60	0.59
query14	1.16	1.16	1.18
query15	0.89	0.85	0.85
query16	0.39	0.38	0.38
query17	1.05	1.00	1.04
query18	0.21	0.20	0.19
query19	1.92	1.80	1.81
query20	0.02	0.01	0.01
query21	15.40	0.93	0.57
query22	0.75	1.19	0.65
query23	14.98	1.37	0.62
query24	7.66	1.65	0.89
query25	0.54	0.19	0.13
query26	0.76	0.16	0.15
query27	0.05	0.05	0.05
query28	9.83	0.89	0.46
query29	12.60	4.04	3.32
query30	0.26	0.08	0.06
query31	2.83	0.60	0.39
query32	3.22	0.54	0.46
query33	3.11	3.06	3.05
query34	15.77	5.14	4.51
query35	4.54	4.52	4.47
query36	0.66	0.49	0.48
query37	0.08	0.06	0.07
query38	0.04	0.04	0.03
query39	0.03	0.02	0.03
query40	0.18	0.14	0.12
query41	0.08	0.02	0.02
query42	0.03	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 105 s
Total hot run time: 29.67 s

morrySnow
morrySnow reviewed Apr 22, 2025
View reviewed changes
Comment threadfe/fe-core/src/main/java/org/apache/doris/httpv2/rest/LoadAction.java
}
}
}
if (needRedirect(request.getScheme())) {

@morrySnowmorrySnowApr 22, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add test case

liaoxin01
liaoxin01 reviewed Apr 23, 2025
View reviewed changes
Comment threadfe/fe-core/src/main/java/org/apache/doris/httpv2/rest/LoadAction.java
@englefly

englefly commented Apr 25, 2025

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Apr 25, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 34109 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cb5ce1bd50fa3ca1a8efa65ed388a60a15318a9b, data reload: false
------ Round 1 ----------------------------------
q1	26272	5075	5012	5012
q2	2077	282	189	189
q3	10442	1251	700	700
q4	10253	993	528	528
q5	8234	2393	2391	2391
q6	188	162	134	134
q7	904	744	625	625
q8	9312	1318	1168	1168
q9	6949	5086	5095	5086
q10	6825	2319	1906	1906
q11	496	294	272	272
q12	349	356	218	218
q13	17771	3663	3074	3074
q14	238	225	217	217
q15	535	478	492	478
q16	456	449	397	397
q17	621	865	381	381
q18	7626	7278	7116	7116
q19	1213	950	560	560
q20	340	343	240	240
q21	4165	3335	2462	2462
q22	1037	1014	955	955
Total cold run time: 116303 ms
Total hot run time: 34109 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5106	5094	5064	5064
q2	247	328	228	228
q3	2171	2647	2319	2319
q4	1435	1858	1482	1482
q5	4593	4506	4404	4404
q6	209	167	127	127
q7	1953	1909	1779	1779
q8	2622	2515	2564	2515
q9	7237	7222	7157	7157
q10	3005	3204	2753	2753
q11	571	512	482	482
q12	689	753	639	639
q13	3598	3809	3279	3279
q14	264	286	271	271
q15	523	487	478	478
q16	469	543	490	490
q17	1172	1549	1379	1379
q18	7681	7515	7321	7321
q19	813	834	904	834
q20	1996	1959	1831	1831
q21	5110	4704	4700	4700
q22	1085	1052	982	982
Total cold run time: 52549 ms
Total hot run time: 50514 ms

@doris-robot

doris-robot commented Apr 25, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 186506 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cb5ce1bd50fa3ca1a8efa65ed388a60a15318a9b, data reload: false
query1	1043	484	509	484
query2	6567	1901	1901	1901
query3	6747	220	218	218
query4	26641	23801	23342	23342
query5	4367	597	461	461
query6	309	202	191	191
query7	4626	477	296	296
query8	294	236	228	228
query9	8598	2593	2592	2592
query10	470	344	256	256
query11	15362	15009	14852	14852
query12	151	112	100	100
query13	1642	511	424	424
query14	8614	6075	6132	6075
query15	203	198	182	182
query16	7126	640	494	494
query17	944	724	577	577
query18	1978	415	302	302
query19	194	190	185	185
query20	119	115	117	115
query21	209	128	107	107
query22	4254	4385	4297	4297
query23	33798	33164	33034	33034
query24	8280	2402	2378	2378
query25	550	490	408	408
query26	1242	272	158	158
query27	2742	502	322	322
query28	4333	2128	2117	2117
query29	816	619	424	424
query30	277	211	188	188
query31	909	848	758	758
query32	74	64	65	64
query33	554	374	314	314
query34	800	874	527	527
query35	786	805	724	724
query36	955	1003	878	878
query37	116	101	74	74
query38	4151	4180	4172	4172
query39	1504	1435	1384	1384
query40	213	123	105	105
query41	56	54	59	54
query42	124	110	106	106
query43	506	523	497	497
query44	1310	816	807	807
query45	189	173	173	173
query46	841	1027	620	620
query47	1746	1771	1686	1686
query48	377	424	298	298
query49	780	500	425	425
query50	659	662	398	398
query51	4090	4130	4046	4046
query52	103	107	100	100
query53	228	255	181	181
query54	591	574	521	521
query55	82	81	80	80
query56	333	303	301	301
query57	1112	1158	1069	1069
query58	263	258	254	254
query59	2820	2815	2636	2636
query60	343	323	308	308
query61	131	125	123	123
query62	807	718	672	672
query63	235	187	187	187
query64	4417	1009	673	673
query65	4372	4286	4321	4286
query66	1161	415	314	314
query67	15798	15657	15272	15272
query68	8085	882	511	511
query69	473	298	270	270
query70	1229	1120	1146	1120
query71	447	316	294	294
query72	5881	4744	4770	4744
query73	714	624	392	392
query74	9300	9054	8697	8697
query75	3813	3171	2699	2699
query76	3639	1204	749	749
query77	810	394	292	292
query78	10054	10232	9338	9338
query79	2045	865	574	574
query80	594	529	467	467
query81	483	256	217	217
query82	473	122	95	95
query83	251	256	232	232
query84	295	111	88	88
query85	776	358	325	325
query86	353	332	283	283
query87	4396	4449	4327	4327
query88	3477	2227	2241	2227
query89	386	312	287	287
query90	1919	208	218	208
query91	137	140	112	112
query92	73	62	57	57
query93	1155	951	585	585
query94	672	415	315	315
query95	376	293	284	284
query96	502	562	275	275
query97	3133	3257	3169	3169
query98	252	203	202	202
query99	1471	1402	1276	1276
Total cold run time: 273696 ms
Total hot run time: 186506 ms

@doris-robot

doris-robot commented Apr 25, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 29.58 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cb5ce1bd50fa3ca1a8efa65ed388a60a15318a9b, data reload: false
query1	0.03	0.03	0.03
query2	0.12	0.10	0.11
query3	0.26	0.19	0.19
query4	1.60	0.19	0.20
query5	0.59	0.58	0.58
query6	1.18	0.72	0.72
query7	0.03	0.02	0.02
query8	0.05	0.04	0.03
query9	0.57	0.54	0.52
query10	0.57	0.58	0.58
query11	0.16	0.11	0.11
query12	0.15	0.11	0.11
query13	0.62	0.60	0.59
query14	1.24	1.19	1.21
query15	0.86	0.85	0.86
query16	0.38	0.39	0.38
query17	1.02	1.04	1.08
query18	0.22	0.19	0.19
query19	1.93	1.77	1.81
query20	0.01	0.01	0.02
query21	15.41	0.89	0.54
query22	0.75	1.22	0.64
query23	14.92	1.38	0.65
query24	7.46	0.92	0.74
query25	0.46	0.16	0.20
query26	0.64	0.17	0.13
query27	0.06	0.04	0.04
query28	9.22	0.86	0.44
query29	12.55	3.91	3.25
query30	0.25	0.09	0.06
query31	2.83	0.59	0.38
query32	3.23	0.54	0.46
query33	3.02	3.02	3.11
query34	15.82	5.07	4.51
query35	4.49	4.57	4.56
query36	0.68	0.50	0.49
query37	0.08	0.06	0.06
query38	0.06	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.14	0.12
query41	0.07	0.02	0.02
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.85 s
Total hot run time: 29.58 s

liaoxin01
liaoxin01 approved these changes Apr 28, 2025
View reviewed changes

@liaoxin01liaoxin01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Apr 28, 2025
@github-actions

github-actionsBot commented Apr 28, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Apr 28, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@englefly
englefly merged commit 3eaa3ac into apache:masterMay 7, 2025
@englefly
englefly deleted the auditStreamloaderHttps branch May 7, 2025 06:27
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@engleflykoarz
[fix](stream loader) do not redirect to https for stream load (apache…
232ddc3
…#50009)
### What problem does this PR solve?
auditlog stream load plugin does not work when https is enabled.
The reason is when https is enabled, LoadAction will add an extra
redirection(from http to https), which breaks the process of audit
plugin.
@engleflyenglefly mentioned this pull request Jun 20, 2025
Merged
16 tasks
morrySnow pushed a commit that referenced this pull request Jun 20, 2025
@englefly
branch-3.1: [fix](stream loader) do not redirect to https for stream …
e5f3c86
…load #50009 (#52046)
pick #50009
@engleflyenglefly mentioned this pull request Jun 26, 2025
Merged
16 tasks
dataroaring pushed a commit that referenced this pull request Jul 8, 2025
@englefly
branch-3.0 [fix](stream loader) do not redirect to https for stream l…
436c04a
…oad #50009 (#52332)
### What problem does this PR solve?
pick #50009
Issue Number: close #xxx
yiguolei added a commit to yiguolei/incubator-doris that referenced this pull request Dec 19, 2025
@yiguolei
branch-2.1 [fix](stream loader) do not redirect to https for stream l…
cfcd42d
…oad apache#50009 (apache#5146)
## Proposed changes
pick apache#50009
Issue Number: close #xxx
<!--Describe your changes.-->
@coderabbitaicoderabbitaiBot mentioned this pull request Aug 20, 2026
Merged
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnowmorrySnowmorrySnow left review comments
@liaoxin01liaoxin01liaoxin01 approved these changes
@starocean999starocean999starocean999 approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.1.xdev/2.1.x-conflictdev/3.0.7-mergeddev/3.1.0-mergedreviewedusercaseImportant user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@englefly@hello-stephen@doris-robot@liaoxin01@starocean999@morrySnow@dataroaring