Skip to content

[fix](vault) fix UnsupportedOperationException if no use_path_style in CreateStorageVaultStmt - #50615

Merged
gavinchou merged 5 commits into
apache:masterfrom
yagagagaga:fix_create_storage_vault_error
May 7, 2025
Merged

[fix](vault) fix UnsupportedOperationException if no use_path_style in CreateStorageVaultStmt#50615
gavinchou merged 5 commits into
apache:masterfrom
yagagagaga:fix_create_storage_vault_error

Conversation

@yagagagaga

@yagagagagayagagagaga commented May 6, 2025•
edited
Loading

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #50356#45155

Problem Summary:

CREATE STORAGE VAULT test PROPERTIES
(
"type"="S3",
"s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
"s3.access_key"="*****",
"s3.secret_key"="*****",
"s3.region"="ap-guangzhou",
"s3.root.path"="prefix",
"s3.bucket"="test",
"provider"="COS"
)

throw Exception:

2025-05-0615:05:16,395WARN (mysql-nio-pool-0|235) [StmtExecutor.handleDdlStmt():3216] DDLstatement(CREATESTORAGEVAULTtestPROPERTIES
(
"type"="S3",
"s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
"s3.access_key" = "*****",
"s3.secret_key" = "*****",
"s3.region" = "ap-guangzhou",
"s3.root.path" = "prefix",
"s3.bucket" = "test",
"provider" = "COS"
)) processfailed.
java.lang.UnsupportedOperationException: nullatcom.google.common.collect.ImmutableMap.put(ImmutableMap.java:814) ~[guava-33.2.1-jre.jar:?]
atorg.apache.doris.catalog.StorageVault.getStorageVaultInstance(StorageVault.java:151) ~[doris-fe.jar:1.2-SNAPSHOT]
atorg.apache.doris.catalog.StorageVault.fromStmt(StorageVault.java:103) ~[doris-fe.jar:1.2-SNAPSHOT]
atorg.apache.doris.catalog.StorageVaultMgr.createStorageVaultResource(StorageVaultMgr.java:69) ~[doris-fe.jar:1.2-SNAPSHOT]
atorg.apache.doris.qe.DdlExecutor.execute(DdlExecutor.java:440) ~[doris-fe.jar:1.2-SNAPSHOT]
atorg.apache.doris.qe.StmtExecutor.handleDdlStmt(StmtExecutor.java:3195) ~[doris-fe.jar:1.2-SNAPSHOT]
atorg.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:1124) ~[doris-fe.jar:1.2-SNAPSHOT]
atorg.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:644) ~[doris-fe.jar:1.2-SNAPSHOT]
atorg.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:574) ~[doris-fe.jar:1.2-SNAPSHOT]
atorg.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:559) ~[doris-fe.jar:1.2-SNAPSHOT]
atorg.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:349) ~[doris-fe.jar:1.2-SNAPSHOT]
atorg.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:249) ~[doris-fe.jar:1.2-SNAPSHOT]
atorg.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:233) ~[doris-fe.jar:1.2-SNAPSHOT]
atorg.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:261) ~[doris-fe.jar:1.2-SNAPSHOT]
atorg.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:444) ~[doris-fe.jar:1.2-SNAPSHOT]
atorg.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
atjava.lang.Thread.run(Thread.java:833) ~[?:?]

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen

hello-stephen commented May 6, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

gavinchou
gavinchou previously approved these changes May 6, 2025
View reviewed changes
gavinchou
gavinchou reviewed May 6, 2025
View reviewed changes
Comment threadfe/fe-core/src/main/java/org/apache/doris/analysis/CreateStorageVaultStmt.java Outdated
this.vaultName = vaultName;
this.properties = ImmutableMap.copyOf(properties);

if (!properties.containsKey(PropertyConverter.USE_PATH_STYLE)) {

@gavinchougavinchouMay 6, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check S3 ?

gavinchou
gavinchou reviewed May 6, 2025
View reviewed changes

@gavinchougavinchou left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should check s3 vault for properties only

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label May 6, 2025
@github-actions

github-actionsBot commented May 6, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented May 6, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@swjtu-zhanglei

swjtu-zhanglei commented May 6, 2025

Copy link
Copy Markdown
Contributor

Add a regression test ?

morningman
morningman previously approved these changes May 6, 2025
View reviewed changes

@morningmanmorningman left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yagagagaga
yagagagaga dismissed stale reviews from morningman and gavinchou via c24c5ccMay 6, 2025 07:30
@github-actionsgithub-actionsBot removed the approved Indicates a PR has been approved by one committer. label May 6, 2025
@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label May 6, 2025
@github-actions

github-actionsBot commented May 6, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@yagagagaga

yagagagaga commented May 6, 2025

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented May 6, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 33816 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 47bf13d632de95aa94ae3d128d68e67b0f6cdbf9, data reload: false
------ Round 1 ----------------------------------
q1	26212	5113	5001	5001
q2	2063	287	193	193
q3	10372	1223	699	699
q4	10233	994	512	512
q5	7545	2320	2357	2320
q6	181	163	134	134
q7	916	772	607	607
q8	9300	1284	1072	1072
q9	6893	5143	5109	5109
q10	6815	2310	1891	1891
q11	476	284	269	269
q12	344	343	220	220
q13	17763	3641	3063	3063
q14	228	227	207	207
q15	527	477	477	477
q16	424	431	369	369
q17	594	857	366	366
q18	7472	7126	7134	7126
q19	1223	928	545	545
q20	337	360	224	224
q21	3935	3338	2459	2459
q22	1047	989	953	953
Total cold run time: 114900 ms
Total hot run time: 33816 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5106	5072	5044	5044
q2	238	326	234	234
q3	2161	2713	2261	2261
q4	1346	1822	1375	1375
q5	4490	4423	4412	4412
q6	220	167	128	128
q7	1999	1962	1726	1726
q8	2651	2531	2511	2511
q9	7119	7089	7190	7089
q10	3043	3163	2720	2720
q11	575	504	483	483
q12	670	748	590	590
q13	3474	3789	3303	3303
q14	284	289	270	270
q15	532	481	480	480
q16	441	491	443	443
q17	1131	1582	1382	1382
q18	7689	7398	7432	7398
q19	801	837	935	837
q20	1998	2006	1892	1892
q21	5129	4645	4487	4487
q22	1004	980	973	973
Total cold run time: 52101 ms
Total hot run time: 50038 ms

@doris-robot

doris-robot commented May 6, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 185734 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 47bf13d632de95aa94ae3d128d68e67b0f6cdbf9, data reload: false
query1	995	479	498	479
query2	6554	1845	1826	1826
query3	6738	216	217	216
query4	25954	23786	23384	23384
query5	4344	624	468	468
query6	294	208	210	208
query7	4625	490	289	289
query8	303	254	243	243
query9	8609	2559	2570	2559
query10	474	326	261	261
query11	15610	15191	14848	14848
query12	155	107	106	106
query13	1686	518	406	406
query14	8820	6205	6041	6041
query15	204	181	172	172
query16	7323	613	456	456
query17	1182	751	571	571
query18	1965	382	293	293
query19	191	179	156	156
query20	119	117	114	114
query21	210	125	107	107
query22	4058	4128	4086	4086
query23	33728	33042	32871	32871
query24	8497	2395	2388	2388
query25	510	452	375	375
query26	1261	265	148	148
query27	2774	490	323	323
query28	4314	2083	2080	2080
query29	783	547	428	428
query30	279	218	187	187
query31	925	858	764	764
query32	72	65	61	61
query33	570	366	318	318
query34	798	844	521	521
query35	772	800	722	722
query36	941	979	882	882
query37	116	103	80	80
query38	4003	4204	4094	4094
query39	1512	1397	1400	1397
query40	217	128	105	105
query41	59	59	55	55
query42	122	108	115	108
query43	498	499	466	466
query44	1253	773	788	773
query45	178	172	166	166
query46	834	1012	633	633
query47	1746	1778	1696	1696
query48	389	410	301	301
query49	745	517	415	415
query50	636	684	399	399
query51	4071	4146	4111	4111
query52	113	105	103	103
query53	225	257	193	193
query54	600	574	525	525
query55	81	80	89	80
query56	324	331	315	315
query57	1154	1125	1105	1105
query58	275	274	261	261
query59	2618	2679	2651	2651
query60	338	337	327	327
query61	126	127	128	127
query62	790	737	657	657
query63	220	180	178	178
query64	4379	988	663	663
query65	4350	4247	4222	4222
query66	1155	408	314	314
query67	15815	15454	15476	15454
query68	7997	878	506	506
query69	465	303	258	258
query70	1222	1108	1082	1082
query71	471	325	303	303
query72	5560	4646	4792	4646
query73	696	570	339	339
query74	9111	9125	8785	8785
query75	3776	3212	2677	2677
query76	3681	1185	739	739
query77	826	375	285	285
query78	9996	10066	9274	9274
query79	2817	815	576	576
query80	639	516	439	439
query81	465	260	228	228
query82	457	125	98	98
query83	280	249	238	238
query84	296	111	87	87
query85	781	346	317	317
query86	334	285	300	285
query87	4373	4369	4292	4292
query88	3234	2214	2223	2214
query89	405	327	282	282
query90	1956	214	213	213
query91	143	138	109	109
query92	84	64	58	58
query93	2473	953	559	559
query94	677	421	310	310
query95	375	296	291	291
query96	488	566	275	275
query97	3124	3258	3151	3151
query98	229	204	203	203
query99	1413	1398	1284	1284
Total cold run time: 274423 ms
Total hot run time: 185734 ms

@doris-robot

doris-robot commented May 6, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 28.96 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 47bf13d632de95aa94ae3d128d68e67b0f6cdbf9, data reload: false
query1	0.03	0.04	0.03
query2	0.13	0.11	0.11
query3	0.26	0.19	0.20
query4	1.61	0.19	0.20
query5	0.58	0.60	0.60
query6	1.19	0.72	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.58	0.51	0.50
query10	0.56	0.57	0.56
query11	0.16	0.11	0.11
query12	0.15	0.11	0.11
query13	0.61	0.60	0.59
query14	0.78	0.80	0.80
query15	0.87	0.86	0.85
query16	0.38	0.38	0.39
query17	1.01	0.99	1.08
query18	0.21	0.20	0.18
query19	1.91	1.75	1.82
query20	0.01	0.01	0.02
query21	15.43	0.90	0.56
query22	0.76	1.28	0.80
query23	14.73	1.37	0.62
query24	7.28	1.48	0.49
query25	0.47	0.22	0.08
query26	0.66	0.17	0.15
query27	0.04	0.05	0.05
query28	9.59	0.88	0.44
query29	12.58	3.95	3.26
query30	0.25	0.09	0.07
query31	2.83	0.59	0.38
query32	3.22	0.54	0.46
query33	3.00	3.13	3.10
query34	15.82	5.07	4.48
query35	4.48	4.55	4.49
query36	0.68	0.49	0.49
query37	0.09	0.06	0.06
query38	0.04	0.04	0.05
query39	0.03	0.02	0.02
query40	0.17	0.13	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 103.39 s
Total hot run time: 28.96 s

@yagagagaga

yagagagaga commented May 6, 2025

Copy link
Copy Markdown
ContributorAuthor

run cloud_p0

@gavinchou
gavinchou merged commit 210b960 into apache:masterMay 7, 2025
yagagagaga added a commit to yagagagaga/doris that referenced this pull request May 7, 2025
@yagagagaga
[fix](vault) fix UnsupportedOperationException if no use_path_style…
4a0a436
… in CreateStorageVaultStmt (apache#50615)
Related PR: apache#50356apache#45155
Problem Summary:
```sql
CREATE STORAGE VAULT test PROPERTIES
(
"type"="S3",
"s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
"s3.access_key" = "*****",
"s3.secret_key" = "*****",
"s3.region" = "ap-guangzhou",
"s3.root.path" = "prefix",
"s3.bucket" = "test",
"provider" = "COS"
)
```
throw Exception:
```java
2025-05-06 15:05:16,395 WARN (mysql-nio-pool-0|235) [StmtExecutor.handleDdlStmt():3216] DDL statement(CREATE STORAGE VAULT test PROPERTIES
(
"type"="S3",
"s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
"s3.access_key" = "*****",
"s3.secret_key" = "*****",
"s3.region" = "ap-guangzhou",
"s3.root.path" = "prefix",
"s3.bucket" = "test",
"provider" = "COS"
)) process failed.
java.lang.UnsupportedOperationException: null
at com.google.common.collect.ImmutableMap.put(ImmutableMap.java:814) ~[guava-33.2.1-jre.jar:?]
at org.apache.doris.catalog.StorageVault.getStorageVaultInstance(StorageVault.java:151) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.StorageVault.fromStmt(StorageVault.java:103) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.StorageVaultMgr.createStorageVaultResource(StorageVaultMgr.java:69) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.DdlExecutor.execute(DdlExecutor.java:440) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.handleDdlStmt(StmtExecutor.java:3195) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:1124) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:644) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:574) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:559) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:349) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:249) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:233) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:261) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:444) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
at java.lang.Thread.run(Thread.java:833) ~[?:?]
```
(cherry picked from commit 210b960)
@yagagagagayagagagaga mentioned this pull request May 7, 2025
Closed
morningman pushed a commit to morningman/doris that referenced this pull request May 7, 2025
@yagagagaga@morningman
[fix](vault) fix UnsupportedOperationException if no use_path_style…
4d25e92
… in CreateStorageVaultStmt (apache#50615)
Related PR: apache#50356apache#45155
Problem Summary:
```sql
CREATE STORAGE VAULT test PROPERTIES
(
"type"="S3",
"s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
"s3.access_key" = "*****",
"s3.secret_key" = "*****",
"s3.region" = "ap-guangzhou",
"s3.root.path" = "prefix",
"s3.bucket" = "test",
"provider" = "COS"
)
```
throw Exception:
```java
2025-05-06 15:05:16,395 WARN (mysql-nio-pool-0|235) [StmtExecutor.handleDdlStmt():3216] DDL statement(CREATE STORAGE VAULT test PROPERTIES
(
"type"="S3",
"s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
"s3.access_key" = "*****",
"s3.secret_key" = "*****",
"s3.region" = "ap-guangzhou",
"s3.root.path" = "prefix",
"s3.bucket" = "test",
"provider" = "COS"
)) process failed.
java.lang.UnsupportedOperationException: null
at com.google.common.collect.ImmutableMap.put(ImmutableMap.java:814) ~[guava-33.2.1-jre.jar:?]
at org.apache.doris.catalog.StorageVault.getStorageVaultInstance(StorageVault.java:151) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.StorageVault.fromStmt(StorageVault.java:103) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.StorageVaultMgr.createStorageVaultResource(StorageVaultMgr.java:69) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.DdlExecutor.execute(DdlExecutor.java:440) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.handleDdlStmt(StmtExecutor.java:3195) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:1124) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:644) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:574) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:559) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:349) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:249) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:233) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:261) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:444) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
at java.lang.Thread.run(Thread.java:833) ~[?:?]
```
@morningmanmorningman mentioned this pull request May 7, 2025
Merged
morningman pushed a commit to morningman/doris that referenced this pull request May 7, 2025
@yagagagaga@morningman
[fix](vault) fix UnsupportedOperationException if no use_path_style…
6c52b12
… in CreateStorageVaultStmt (apache#50615)
Related PR: apache#50356apache#45155
Problem Summary:
```sql
CREATE STORAGE VAULT test PROPERTIES
(
"type"="S3",
"s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
"s3.access_key" = "*****",
"s3.secret_key" = "*****",
"s3.region" = "ap-guangzhou",
"s3.root.path" = "prefix",
"s3.bucket" = "test",
"provider" = "COS"
)
```
throw Exception:
```java
2025-05-06 15:05:16,395 WARN (mysql-nio-pool-0|235) [StmtExecutor.handleDdlStmt():3216] DDL statement(CREATE STORAGE VAULT test PROPERTIES
(
"type"="S3",
"s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
"s3.access_key" = "*****",
"s3.secret_key" = "*****",
"s3.region" = "ap-guangzhou",
"s3.root.path" = "prefix",
"s3.bucket" = "test",
"provider" = "COS"
)) process failed.
java.lang.UnsupportedOperationException: null
at com.google.common.collect.ImmutableMap.put(ImmutableMap.java:814) ~[guava-33.2.1-jre.jar:?]
at org.apache.doris.catalog.StorageVault.getStorageVaultInstance(StorageVault.java:151) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.StorageVault.fromStmt(StorageVault.java:103) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.StorageVaultMgr.createStorageVaultResource(StorageVaultMgr.java:69) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.DdlExecutor.execute(DdlExecutor.java:440) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.handleDdlStmt(StmtExecutor.java:3195) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:1124) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:644) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:574) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:559) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:349) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:249) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:233) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:261) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:444) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
at java.lang.Thread.run(Thread.java:833) ~[?:?]
```
@yagagagaga
yagagagaga deleted the fix_create_storage_vault_error branch May 8, 2025 06:32
morningman pushed a commit to morningman/doris that referenced this pull request May 14, 2025
@yagagagaga@morningman
[fix](vault) fix UnsupportedOperationException if no use_path_style…
d3608b4
… in CreateStorageVaultStmt (apache#50615)
Related PR: apache#50356apache#45155
Problem Summary:
```sql
CREATE STORAGE VAULT test PROPERTIES
(
"type"="S3",
"s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
"s3.access_key" = "*****",
"s3.secret_key" = "*****",
"s3.region" = "ap-guangzhou",
"s3.root.path" = "prefix",
"s3.bucket" = "test",
"provider" = "COS"
)
```
throw Exception:
```java
2025-05-06 15:05:16,395 WARN (mysql-nio-pool-0|235) [StmtExecutor.handleDdlStmt():3216] DDL statement(CREATE STORAGE VAULT test PROPERTIES
(
"type"="S3",
"s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
"s3.access_key" = "*****",
"s3.secret_key" = "*****",
"s3.region" = "ap-guangzhou",
"s3.root.path" = "prefix",
"s3.bucket" = "test",
"provider" = "COS"
)) process failed.
java.lang.UnsupportedOperationException: null
at com.google.common.collect.ImmutableMap.put(ImmutableMap.java:814) ~[guava-33.2.1-jre.jar:?]
at org.apache.doris.catalog.StorageVault.getStorageVaultInstance(StorageVault.java:151) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.StorageVault.fromStmt(StorageVault.java:103) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.StorageVaultMgr.createStorageVaultResource(StorageVaultMgr.java:69) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.DdlExecutor.execute(DdlExecutor.java:440) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.handleDdlStmt(StmtExecutor.java:3195) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:1124) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:644) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:574) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:559) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:349) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:249) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:233) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:261) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:444) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
at java.lang.Thread.run(Thread.java:833) ~[?:?]
```
dataroaring pushed a commit that referenced this pull request May 15, 2025
@morningman@yagagagaga
branch-3.0: [fix](create-resource) fix potential concurrent modificat…
3ae4211
…ion exception when creating resource (#50356, #50615) (#50533)
bp #50356#50615
---------
Co-authored-by: yagagagaga <zhangminke@selectdb.com>
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@yagagagagakoarz
[fix](vault) fix UnsupportedOperationException if no use_path_style…
dc3c412
… in CreateStorageVaultStmt (apache#50615)
Related PR: apache#50356apache#45155
Problem Summary:
```sql
CREATE STORAGE VAULT test PROPERTIES
(
"type"="S3",
"s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
"s3.access_key" = "*****",
"s3.secret_key" = "*****",
"s3.region" = "ap-guangzhou",
"s3.root.path" = "prefix",
"s3.bucket" = "test",
"provider" = "COS"
)
```
throw Exception:
```java
2025-05-06 15:05:16,395 WARN (mysql-nio-pool-0|235) [StmtExecutor.handleDdlStmt():3216] DDL statement(CREATE STORAGE VAULT test PROPERTIES
(
"type"="S3",
"s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
"s3.access_key" = "*****",
"s3.secret_key" = "*****",
"s3.region" = "ap-guangzhou",
"s3.root.path" = "prefix",
"s3.bucket" = "test",
"provider" = "COS"
)) process failed.
java.lang.UnsupportedOperationException: null
at com.google.common.collect.ImmutableMap.put(ImmutableMap.java:814) ~[guava-33.2.1-jre.jar:?]
at org.apache.doris.catalog.StorageVault.getStorageVaultInstance(StorageVault.java:151) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.StorageVault.fromStmt(StorageVault.java:103) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.StorageVaultMgr.createStorageVaultResource(StorageVaultMgr.java:69) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.DdlExecutor.execute(DdlExecutor.java:440) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.handleDdlStmt(StmtExecutor.java:3195) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:1124) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:644) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:574) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:559) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:349) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:249) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:233) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:261) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:444) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
at java.lang.Thread.run(Thread.java:833) ~[?:?]
```
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningmanmorningmanmorningman approved these changes
@gavinchougavinchougavinchou approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/3.0.6-mergedp0_testreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@yagagagaga@hello-stephen@swjtu-zhanglei@doris-robot@morningman@gavinchou@dataroaring