[chore](dep)Exclude AWS SDK bundle to reduce dependency size and include only required S3 modules by CalvinKirs · Pull Request #57629 · apache/doris

CalvinKirs · 2025-11-03T07:37:56Z

…

This PR excludes the aws-java-sdk-bundle from the hadoop-aws dependency to reduce the overall project size by including only the necessary S3 SDK modules.

…ude only required S3 modules This PR excludes the aws-java-sdk-bundle from the hadoop-aws dependency to reduce the overall project size by including only the necessary S3 SDK modules.

hello-stephen · 2025-11-03T07:38:02Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

CalvinKirs · 2025-11-03T07:38:09Z

run buildall

github-actions · 2025-11-03T07:47:31Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-11-03T07:47:34Z

PR approved by anyone and no changes requested.

doris-robot · 2025-11-03T08:28:39Z

ClickBench: Total hot run time: 28.86 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 017a05dc1f4561315b2e8b1c00256d743d0eda7b, data reload: false
query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.25	0.09	0.09
query4	1.60	0.12	0.12
query5	0.30	0.27	0.27
query6	1.20	0.67	0.68
query7	0.03	0.02	0.02
query8	0.05	0.05	0.05
query9	0.65	0.57	0.56
query10	0.62	0.62	0.61
query11	0.18	0.12	0.13
query12	0.17	0.13	0.14
query13	0.63	0.61	0.60
query14	1.03	1.03	1.03
query15	0.92	0.86	0.89
query16	0.43	0.42	0.42
query17	1.12	1.17	1.21
query18	0.23	0.22	0.21
query19	2.02	1.90	1.89
query20	0.02	0.01	0.01
query21	15.36	0.22	0.15
query22	4.95	0.08	0.06
query23	15.64	0.29	0.12
query24	1.75	1.62	0.35
query25	0.09	0.06	0.06
query26	0.17	0.16	0.15
query27	0.07	0.07	0.07
query28	3.96	1.20	0.96
query29	12.62	4.48	3.75
query30	0.31	0.15	0.12
query31	2.82	0.68	0.42
query32	3.25	0.58	0.48
query33	3.21	3.20	3.18
query34	15.77	5.24	4.57
query35	4.67	4.60	4.58
query36	0.73	0.53	0.52
query37	0.12	0.07	0.07
query38	0.07	0.05	0.05
query39	0.04	0.04	0.03
query40	0.18	0.15	0.14
query41	0.10	0.03	0.04
query42	0.05	0.03	0.03
query43	0.05	0.05	0.04
Total cold run time: 97.59 s
Total hot run time: 28.86 s

CalvinKirs · 2025-11-03T08:49:44Z

run buildall

doris-robot · 2025-11-03T09:39:30Z

TPC-DS: Total hot run time: 189703 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fc79ae060e282c43106e44df311241804feb7da4, data reload: false
query1	1060	396	389	389
query2	6549	1686	1661	1661
query3	6753	221	215	215
query4	26334	23741	23800	23741
query5	5838	631	458	458
query6	339	236	237	236
query7	4667	499	296	296
query8	325	272	261	261
query9	8712	2585	2582	2582
query10	552	351	294	294
query11	15780	15099	14806	14806
query12	181	117	114	114
query13	1704	557	451	451
query14	11998	9293	9361	9293
query15	206	197	174	174
query16	7693	685	521	521
query17	1868	812	701	701
query18	2211	484	373	373
query19	255	240	212	212
query20	149	145	173	145
query21	234	155	129	129
query22	4901	4828	4794	4794
query23	34345	33894	33537	33537
query24	8454	2611	2542	2542
query25	625	530	500	500
query26	1376	283	161	161
query27	2767	519	374	374
query28	4416	2234	2273	2234
query29	826	654	486	486
query30	308	236	209	209
query31	914	831	793	793
query32	80	116	75	75
query33	585	466	366	366
query34	1539	877	523	523
query35	818	821	749	749
query36	978	1069	929	929
query37	137	109	98	98
query38	3637	3573	3476	3476
query39	1483	1397	1431	1397
query40	218	122	111	111
query41	60	55	55	55
query42	118	109	112	109
query43	488	486	476	476
query44	1237	729	728	728
query45	179	178	175	175
query46	881	988	639	639
query47	1751	1793	1726	1726
query48	392	421	304	304
query49	767	505	389	389
query50	643	684	408	408
query51	3931	3871	3869	3869
query52	107	112	96	96
query53	233	264	197	197
query54	306	291	273	273
query55	86	84	83	83
query56	321	318	298	298
query57	1196	1233	1124	1124
query58	293	284	277	277
query59	2610	2637	2622	2622
query60	347	345	329	329
query61	167	159	158	158
query62	780	732	685	685
query63	225	191	191	191
query64	4419	1181	871	871
query65	4035	3943	3955	3943
query66	1036	453	330	330
query67	15048	15037	14841	14841
query68	8591	930	588	588
query69	518	335	291	291
query70	1343	1240	1311	1240
query71	531	343	322	322
query72	6011	4909	4983	4909
query73	673	577	357	357
query74	8927	8915	8739	8739
query75	4141	3326	2821	2821
query76	3882	1163	728	728
query77	836	416	309	309
query78	9632	9730	8820	8820
query79	2207	799	588	588
query80	633	558	501	501
query81	504	263	230	230
query82	492	156	126	126
query83	259	264	242	242
query84	258	106	92	92
query85	886	482	429	429
query86	335	322	301	301
query87	3678	3739	3606	3606
query88	3706	2243	2258	2243
query89	377	324	296	296
query90	2030	216	207	207
query91	168	172	141	141
query92	86	69	70	69
query93	1804	982	633	633
query94	689	454	339	339
query95	400	322	312	312
query96	477	561	277	277
query97	2908	2961	2880	2880
query98	236	208	215	208
query99	1375	1425	1257	1257
Total cold run time: 282064 ms
Total hot run time: 189703 ms

doris-robot · 2025-11-03T09:44:39Z

ClickBench: Total hot run time: 28.27 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fc79ae060e282c43106e44df311241804feb7da4, data reload: false
query1	0.06	0.05	0.04
query2	0.09	0.04	0.04
query3	0.25	0.08	0.08
query4	1.60	0.11	0.11
query5	0.27	0.25	0.26
query6	1.18	0.64	0.64
query7	0.03	0.02	0.03
query8	0.05	0.04	0.04
query9	0.60	0.52	0.51
query10	0.57	0.57	0.58
query11	0.16	0.12	0.11
query12	0.15	0.12	0.13
query13	0.62	0.60	0.60
query14	1.00	1.01	1.01
query15	0.84	0.83	0.84
query16	0.38	0.44	0.41
query17	1.02	1.05	1.03
query18	0.22	0.20	0.21
query19	1.88	1.76	1.82
query20	0.02	0.01	0.01
query21	15.44	0.18	0.13
query22	5.10	0.06	0.05
query23	15.70	0.27	0.10
query24	2.58	1.25	1.34
query25	0.08	0.07	0.07
query26	0.13	0.14	0.13
query27	0.07	0.05	0.06
query28	5.72	1.17	0.93
query29	12.55	3.85	3.20
query30	0.30	0.13	0.12
query31	2.81	0.58	0.38
query32	3.23	0.55	0.47
query33	3.09	3.03	3.03
query34	15.79	5.18	4.55
query35	4.57	4.60	4.56
query36	0.70	0.50	0.50
query37	0.10	0.07	0.07
query38	0.07	0.04	0.03
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.10	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.04
Total cold run time: 99.41 s
Total hot run time: 28.27 s

hello-stephen · 2025-11-03T10:06:54Z

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

github-actions · 2025-11-03T11:23:50Z

PR approved by at least one committer and no changes requested.

…ude only required S3 modules (#57629) … This PR excludes the aws-java-sdk-bundle from the hadoop-aws dependency to reduce the overall project size by including only the necessary S3 SDK modules.

…ize and include only required S3 modules #57629 (#57648) Cherry-picked from #57629 Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>

…ude only required S3 modules (#57629) … This PR excludes the aws-java-sdk-bundle from the hadoop-aws dependency to reduce the overall project size by including only the necessary S3 SDK modules.

…ude only required S3 modules (apache#57629) … This PR excludes the aws-java-sdk-bundle from the hadoop-aws dependency to reduce the overall project size by including only the necessary S3 SDK modules.

…ude only required S3 modules (#57629) … This PR excludes the aws-java-sdk-bundle from the hadoop-aws dependency to reduce the overall project size by including only the necessary S3 SDK modules. (cherry picked from commit 022dce9)

…57782) Cherry-picked from #57710 #57979 #57810 #57784 #57202 #57629 --------- Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>

[chore](dep)Exclude AWS SDK bundle to reduce dependency size and incl…
017a05d
…ude only required S3 modules This PR excludes the aws-java-sdk-bundle from the hadoop-aws dependency to reduce the overall project size by including only the necessary S3 SDK modules.

CalvinKirs requested a review from morningman as a code owner November 3, 2025 07:37

morningman added dev/3.1.x dev/4.0.x labels Nov 3, 2025

morningman previously approved these changes Nov 3, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Nov 3, 2025

github-actionsBot added the reviewed label Nov 3, 2025

fix
fc79ae0

CalvinKirs dismissed morningman’s stale review via fc79ae0November 3, 2025 08:49

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Nov 3, 2025

morningman approved these changes Nov 3, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Nov 3, 2025

hubgeter approved these changes Nov 3, 2025
View reviewed changes

CalvinKirs merged commit 022dce9 into apache:masterNov 3, 2025
27 of 29 checks passed

github-actionsBot mentioned this pull request Nov 3, 2025
branch-3.1: [chore](dep)Exclude AWS SDK bundle to reduce dependency size and include only required S3 modules #57629 #57648
Merged

github-actionsBot mentioned this pull request Nov 3, 2025
branch-4.0: [chore](dep)Exclude AWS SDK bundle to reduce dependency size and include only required S3 modules #57629 #57649
Closed

morrySnow added the dev/3.1.x-conflict label Nov 4, 2025

CalvinKirs removed the dev/3.1.x-conflict label Nov 5, 2025

morrySnow pushed a commit that referenced this pull request Nov 6, 2025
branch-3.1: [chore](dep)Exclude AWS SDK bundle to reduce dependency s…
219a3b8
…ize and include only required S3 modules #57629 (#57648) Cherry-picked from #57629 Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>

morrySnow added dev/3.1.3-merged and removed dev/3.1.x labels Nov 6, 2025

CalvinKirs mentioned this pull request Nov 19, 2025
branch-4.0: [feat](web)Adapted for Spring Boot 3 and Jetty 12 #57710 #57782
Merged

yiguolei pushed a commit that referenced this pull request Nov 20, 2025
branch-4.0: [feat](web)Adapted for Spring Boot 3 and Jetty 12#57710(#…
ff742c4
…57782) Cherry-picked from #57710 #57979 #57810 #57784 #57202 #57629 --------- Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>

yiguolei added dev/4.0.2-merged and removed dev/4.0.x labels Nov 20, 2025

Navigation Menu

Uh oh!

[chore](dep)Exclude AWS SDK bundle to reduce dependency size and include only required S3 modules - #57629

[chore](dep)Exclude AWS SDK bundle to reduce dependency size and include only required S3 modules#57629
CalvinKirs merged 2 commits into
apache:masterfrom
CalvinKirs:master-aws-bundle

CalvinKirs commented Nov 3, 2025

Uh oh!

hello-stephen commented Nov 3, 2025

Uh oh!

CalvinKirs commented Nov 3, 2025

Uh oh!

github-actionsBot commented Nov 3, 2025

Uh oh!

github-actionsBot commented Nov 3, 2025

Uh oh!

doris-robot commented Nov 3, 2025

Uh oh!

CalvinKirs commented Nov 3, 2025

Uh oh!

doris-robot commented Nov 3, 2025

Uh oh!

doris-robot commented Nov 3, 2025

Uh oh!

hello-stephen commented Nov 3, 2025

Uh oh!

github-actionsBot commented Nov 3, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Uh oh!

Conversation

CalvinKirs commented Nov 3, 2025

Uh oh!

hello-stephen commented Nov 3, 2025

Uh oh!

CalvinKirs commented Nov 3, 2025

Uh oh!

github-actionsBot commented Nov 3, 2025

Uh oh!

github-actionsBot commented Nov 3, 2025

Uh oh!

doris-robot commented Nov 3, 2025

Uh oh!

CalvinKirs commented Nov 3, 2025

Uh oh!

doris-robot commented Nov 3, 2025

Uh oh!

doris-robot commented Nov 3, 2025

Uh oh!

hello-stephen commented Nov 3, 2025

FE UT Coverage Report

Uh oh!

github-actionsBot commented Nov 3, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants