Skip to content

branch-3.0 [fix](stream loader) do not redirect to https for stream load #50009 - #52332

Merged
dataroaring merged 3 commits into
apache:branch-3.0from
englefly:pick-https-50009-3.0
Jul 8, 2025
Merged

branch-3.0 [fix](stream loader) do not redirect to https for stream load #50009#52332
dataroaring merged 3 commits into
apache:branch-3.0from
englefly:pick-https-50009-3.0

Conversation

@englefly

@engleflyenglefly commented Jun 26, 2025

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

pick #50009
Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@englefly
[opt](nereids) avoid printing duplicated table stats in profile (apac…
027df92
…he#51354)
### What problem does this PR solve?
Query related tables are collected by function
org.apache.doris.common.profile.Profile#updateSummary().
This function is invoked multiple times during a query execution. And
hence, tables are added to Profile.physicalRelations multiple times.
In this pr, Profile.physicalRelations is defined as Set instead of List
to remove duplications.
(cherry picked from commit 8e41d4d)
@englefly
pick
b560897
@hello-stephen

hello-stephen commented Jun 26, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@englefly
englefly changed the base branch from master to branch-3.0June 26, 2025 02:50
@englefly

englefly commented Jun 26, 2025

Copy link
Copy Markdown
ContributorAuthor

run buildall

@dataroaring

dataroaring commented Jun 27, 2025

Copy link
Copy Markdown
Contributor

run buildall

@doris-robot

doris-robot commented Jun 27, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 40014 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fc922de6836b998b75b9c193ed8b90dcd551d734, data reload: false
------ Round 1 ----------------------------------
q1	17571	6893	6625	6625
q2	2050	161	160	160
q3	10564	1139	1144	1139
q4	10229	752	756	752
q5	7745	2938	2831	2831
q6	219	137	131	131
q7	1014	617	594	594
q8	9349	1975	2039	1975
q9	6640	6448	6473	6448
q10	6996	2282	2279	2279
q11	446	257	265	257
q12	393	216	210	210
q13	17784	2978	3014	2978
q14	232	208	203	203
q15	503	471	457	457
q16	483	388	376	376
q17	971	550	591	550
q18	7362	6584	6560	6560
q19	1395	1145	1095	1095
q20	476	203	195	195
q21	3904	3224	3260	3224
q22	1086	975	990	975
Total cold run time: 107412 ms
Total hot run time: 40014 ms
----- Round 2, with runtime_filter_mode=off -----
q1	6580	6742	6594	6594
q2	329	230	239	230
q3	2910	2790	2897	2790
q4	2183	1891	1796	1796
q5	5665	5724	5745	5724
q6	210	124	125	124
q7	2183	1756	1796	1756
q8	3351	3517	3585	3517
q9	8850	8923	8904	8904
q10	3607	3514	3639	3514
q11	604	486	497	486
q12	785	574	604	574
q13	6131	3152	3160	3152
q14	301	291	281	281
q15	497	482	477	477
q16	500	447	451	447
q17	1828	1624	1611	1611
q18	8192	7766	7636	7636
q19	1703	1636	1605	1605
q20	2117	1903	1885	1885
q21	5105	4930	4840	4840
q22	1099	967	992	967
Total cold run time: 64730 ms
Total hot run time: 58910 ms

@doris-robot

doris-robot commented Jun 27, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 190963 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fc922de6836b998b75b9c193ed8b90dcd551d734, data reload: false
query1	974	376	356	356
query2	6513	1907	1840	1840
query3	6705	216	230	216
query4	34147	23830	23814	23814
query5	4375	466	447	447
query6	272	177	179	177
query7	4656	311	309	309
query8	280	226	219	219
query9	9664	2566	2567	2566
query10	478	273	256	256
query11	18192	15249	15186	15186
query12	160	100	102	100
query13	1648	435	410	410
query14	9734	7115	7451	7115
query15	252	171	185	171
query16	8185	467	487	467
query17	1674	579	562	562
query18	2145	305	321	305
query19	367	157	151	151
query20	115	110	105	105
query21	204	105	105	105
query22	4468	4202	4174	4174
query23	34348	33883	33470	33470
query24	11682	2836	2812	2812
query25	670	415	400	400
query26	1655	167	170	167
query27	2873	355	333	333
query28	7470	2114	2117	2114
query29	991	452	447	447
query30	325	169	158	158
query31	1057	790	792	790
query32	96	59	60	59
query33	788	311	315	311
query34	916	502	505	502
query35	874	713	726	713
query36	1101	938	960	938
query37	136	71	73	71
query38	3974	3885	3817	3817
query39	1525	1434	1440	1434
query40	283	109	104	104
query41	52	50	49	49
query42	117	104	102	102
query43	535	476	490	476
query44	1215	804	783	783
query45	186	170	170	170
query46	1188	706	708	706
query47	1945	1858	1857	1857
query48	481	382	381	381
query49	1233	396	381	381
query50	822	421	415	415
query51	7336	7160	7022	7022
query52	101	90	88	88
query53	259	184	187	184
query54	1214	453	462	453
query55	77	80	82	80
query56	252	246	253	246
query57	1328	1192	1160	1160
query58	232	208	222	208
query59	3010	2826	2791	2791
query60	290	260	249	249
query61	114	104	110	104
query62	892	673	680	673
query63	219	185	198	185
query64	5248	658	634	634
query65	3321	3182	3199	3182
query66	1412	320	305	305
query67	16149	15662	15692	15662
query68	3962	592	579	579
query69	413	259	280	259
query70	1143	1082	1145	1082
query71	323	254	256	254
query72	6185	4175	3998	3998
query73	759	354	347	347
query74	9959	9298	9318	9298
query75	3411	2649	2669	2649
query76	2723	1086	1036	1036
query77	433	273	273	273
query78	10387	9766	9567	9567
query79	1305	594	599	594
query80	819	430	447	430
query81	507	213	216	213
query82	1238	90	84	84
query83	226	143	142	142
query84	246	79	77	77
query85	1099	315	294	294
query86	311	284	282	282
query87	4366	4230	4236	4230
query88	3442	2394	2360	2360
query89	408	294	298	294
query90	2069	184	187	184
query91	182	150	145	145
query92	62	51	55	51
query93	1077	565	561	561
query94	777	294	290	290
query95	355	260	251	251
query96	616	284	278	278
query97	3423	3131	3169	3131
query98	213	197	195	195
query99	1541	1298	1333	1298
Total cold run time: 299180 ms
Total hot run time: 190963 ms

@doris-robot

doris-robot commented Jun 27, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 29.18 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fc922de6836b998b75b9c193ed8b90dcd551d734, data reload: false
query1	0.03	0.04	0.03
query2	0.07	0.03	0.02
query3	0.24	0.07	0.06
query4	1.61	0.10	0.10
query5	0.54	0.49	0.52
query6	1.12	0.73	0.75
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.57	0.50	0.50
query10	0.55	0.54	0.56
query11	0.14	0.10	0.11
query12	0.14	0.12	0.11
query13	0.61	0.62	0.59
query14	0.77	0.80	0.82
query15	0.85	0.83	0.82
query16	0.38	0.39	0.37
query17	1.05	1.04	1.03
query18	0.22	0.21	0.23
query19	1.90	1.87	1.76
query20	0.02	0.01	0.01
query21	15.43	0.60	0.59
query22	2.17	2.27	1.55
query23	16.82	0.95	0.84
query24	3.65	0.34	1.60
query25	0.23	0.11	0.12
query26	0.48	0.14	0.15
query27	0.04	0.04	0.03
query28	10.16	0.54	0.45
query29	12.61	3.18	3.21
query30	0.24	0.06	0.05
query31	2.87	0.38	0.41
query32	3.24	0.47	0.45
query33	3.02	3.01	3.05
query34	17.18	4.49	4.47
query35	4.59	4.51	4.54
query36	0.66	0.47	0.47
query37	0.08	0.06	0.06
query38	0.05	0.03	0.03
query39	0.04	0.02	0.03
query40	0.15	0.14	0.13
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 104.71 s
Total hot run time: 29.18 s

@englefly

englefly commented Jun 30, 2025

Copy link
Copy Markdown
ContributorAuthor

run external

@engleflyenglefly changed the title [fix](stream loader) do not redirect to https for stream load #50009 br-3.0branch-3.0 [fix](stream loader) do not redirect to https for stream load #50009Jul 4, 2025
dataroaring
dataroaring approved these changes Jul 8, 2025
View reviewed changes

@dataroaringdataroaring left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring
dataroaring merged commit 436c04a into apache:branch-3.0Jul 8, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaringdataroaringdataroaring approved these changes
@gavinchougavinchouAwaiting requested review from gavinchou
@w41terw41terAwaiting requested review from w41ter
@morningmanmorningmanAwaiting requested review from morningman
@yiguoleiyiguoleiAwaiting requested review from yiguolei
@xiaokangxiaokangAwaiting requested review from xiaokang
@CalvinKirsCalvinKirsAwaiting requested review from CalvinKirs
@BiteTheDDDDtBiteTheDDDDtAwaiting requested review from BiteTheDDDDt

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@englefly@hello-stephen@dataroaring@doris-robot