Skip to content

branch-4.0: [opt](catalog) support nested namespaces of iceberg #56415 - #56695

Merged
yiguolei merged 1 commit into
branch-4.0from
auto-pick-56415-branch-4.0
Oct 1, 2025
Merged

branch-4.0: [opt](catalog) support nested namespaces of iceberg #56415#56695
yiguolei merged 1 commit into
branch-4.0from
auto-pick-56415-branch-4.0

Conversation

@github-actions

@github-actionsgithub-actionsBot commented Oct 1, 2025

Copy link
Copy Markdown
Contributor

Cherry-picked from #56415

@morningmanYour Name
[opt](catalog) support nested namespaces of iceberg (#56415)
50ce1e8
### What problem does this PR solve?
Iceberg has 3 levels of metadata: catalog, namespace and table, mapping
to Doris' catalog, database and table.
Iceberg support nested namespaces, which means the following namespaces
are valid:
```
ns1
ns1.ns2
ns1.ns2.ns3
```
So we need to support mapping nested namespace to Doris' database.
This PR add a global variable `enable_nested_namespace` to control this
behavior.
Default is `false`, and no logic is changed.
If set to true, Doris can support following statments:
```
mysql> switch iceberg;
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| nested |
| nested.db1 |
| nested.db2 |
+--------------------+
mysql> use iceberg.nested.db1;
ERROR 1049 (42000): Only one dot can be in the name: iceberg.nested.db1
mysql> use iceberg.`nested.db1`;
ERROR 5086 (42000): errCode = 2, detailMessage = Unknown catalog 'nested'
mysql> set global enable_nested_namespace=true;
mysql> use iceberg.nested.db1;
Database changed
mysql> select k1 from iceberg.`nested.db1`.nested1;
mysql> select nested1.k1 from `nested.db1`.nested1;
mysql> select `nested.db1`.nested1.k1 from iceberg.`nested.db1`.nested1;
mysql> select iceberg.`nested.db1`.nested1.k1 from nested1;
+------+
| k1 |
+------+
| 1 |
+------+
mysql> refresh catalog iceberg;
mysql> refresh database iceberg.`nested.db1`;
mysql> refresh table iceberg.`nested.db1`.nested1;
Query OK, 0 rows affected (0.01 sec)
```
But, I can execute statement like:
```
use iceberg.`nested.db1`;
```
I don't know why, there is a very strange behavior in MySQL client, when
adding back quota,
the INIT_DB command can only receive `nested.db1` part, but expect
`iceberg.nested.db1`.
Also support creating nested database name in internal catalog:
```
create database `db1.db2`
```
@hello-stephen

hello-stephen commented Oct 1, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaringdataroaring reopened this Oct 1, 2025
@hello-stephen

hello-stephen commented Oct 1, 2025

Copy link
Copy Markdown
Contributor

run buildall

@hello-stephen

hello-stephen commented Oct 1, 2025

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

@doris-robot

doris-robot commented Oct 1, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.36 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 50ce1e867501a18eb6a27192ef42095608cb1a33, data reload: true
query1	0.04	0.04	0.04
query2	0.11	0.04	0.04
query3	0.28	0.08	0.08
query4	1.74	0.11	0.12
query5	0.26	0.25	0.25
query6	1.29	0.66	0.67
query7	0.03	0.02	0.02
query8	0.06	0.05	0.04
query9	0.60	0.52	0.52
query10	0.58	0.58	0.56
query11	0.19	0.11	0.11
query12	0.16	0.12	0.12
query13	0.66	0.62	0.62
query14	0.80	0.84	0.84
query15	0.89	0.88	0.85
query16	0.41	0.41	0.40
query17	1.11	1.09	1.06
query18	0.20	0.19	0.20
query19	1.95	1.84	1.79
query20	0.02	0.01	0.01
query21	15.46	0.94	0.58
query22	0.81	1.32	0.89
query23	14.89	1.40	0.58
query24	16.29	0.46	0.17
query25	0.20	0.05	0.06
query26	0.33	0.16	0.14
query27	0.05	0.05	0.06
query28	11.21	1.00	0.93
query29	12.73	3.93	3.24
query30	0.30	0.13	0.11
query31	3.01	0.61	0.39
query32	3.31	0.60	0.50
query33	3.42	3.36	3.35
query34	17.17	5.87	5.02
query35	5.09	5.09	5.13
query36	0.75	0.58	0.56
query37	0.10	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.04	0.04
query40	0.18	0.16	0.16
query41	0.09	0.03	0.03
query42	0.05	0.04	0.03
query43	0.04	0.04	0.04
Total cold run time: 116.95 s
Total hot run time: 30.36 s

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Oct 1, 2025
@github-actions

github-actionsBot commented Oct 1, 2025

Copy link
Copy Markdown
ContributorAuthor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Oct 1, 2025

Copy link
Copy Markdown
ContributorAuthor

PR approved by anyone and no changes requested.

@yiguolei
yiguolei merged commit 1ff1654 into branch-4.0Oct 1, 2025
24 of 26 checks passed
@morningman
morningman deleted the auto-pick-56415-branch-4.0 branch July 8, 2026 14:08
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguoleiyiguoleiyiguolei approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@hello-stephen@doris-robot@yiguolei@dataroaring@morningman