Skip to content

[feature](information_schema)add metadata_name_ids for quickly get c… - #23980

Merged
xiaokang merged 2 commits into
apache:branch-2.0from
hubgeter:pick_tables_2.0
Sep 6, 2023
Merged

[feature](information_schema)add metadata_name_ids for quickly get c…#23980
xiaokang merged 2 commits into
apache:branch-2.0from
hubgeter:pick_tables_2.0

Conversation

@hubgeter

Copy link
Copy Markdown
Contributor

…atlogs,db,table and add profiling table in order to Compatible with mysql (#22702)

add information_schema.metadata_name_idsfor quickly get catlogs,db,table.

  1. table struct :
mysql>descinternal.information_schema.metadata_name_ids;
+---------------+--------------+------+-------+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------------+--------------+------+-------+---------+-------+
| CATALOG_ID | BIGINT | Yes | false | NULL | |
| CATALOG_NAME | VARCHAR(512) | Yes | false | NULL | |
| DATABASE_ID | BIGINT | Yes | false | NULL | |
| DATABASE_NAME | VARCHAR(64) | Yes | false | NULL | |
| TABLE_ID | BIGINT | Yes | false | NULL | |
| TABLE_NAME | VARCHAR(64) | Yes | false | NULL | |
+---------------+--------------+------+-------+---------+-------+6 rows inset (0.00 sec)
mysql>select*frominternal.information_schema.metadata_name_ids where CATALOG_NAME="hive1"limit1 \G;
***************************1. row ***************************
CATALOG_ID: 113008
CATALOG_NAME: hive1
DATABASE_ID: 113042
DATABASE_NAME: ssb1_parquet
TABLE_ID: 114009
TABLE_NAME: dates
1 row inset (0.07 sec)
  1. when you create / drop catalog , need not refresh catalog .
mysql>selectcount(*) frominternal.information_schema.metadata_name_ids\G;
***************************1. row ***************************count(*): 213011 row inset (0.34 sec)
mysql> drop catalog hive2;
Query OK, 0 rows affected (0.01 sec)
mysql>selectcount(*) frominternal.information_schema.metadata_name_ids\G;
***************************1. row ***************************count(*): 106651 row inset (0.04 sec)
mysql> create catalog hive3 ...
mysql>selectcount(*) frominternal.information_schema.metadata_name_ids\G;
***************************1. row ***************************count(*): 213011 row inset (0.32 sec)
  1. create / drop table , need not refresh catalog .
mysql> CREATE TABLE IF NOT EXISTS demo.example_tbl ... ;
mysql>selectcount(*) frominternal.information_schema.metadata_name_ids\G;
***************************1. row ***************************count(*): 106661 row inset (0.04 sec)
mysql>droptabledemo.example_tbl;
Query OK, 0 rows affected (0.01 sec)
mysql>selectcount(*) frominternal.information_schema.metadata_name_ids\G;
***************************1. row ***************************count(*): 106651 row inset (0.04 sec)
  1. you can set query time , prevent queries from taking too long .

fe.conf : query_metadata_name_ids_timeout
the time used to obtain all tables in one database
  1. add information_schema.profiling in order to Compatible with mysql
mysql>select*frominformation_schema.profiling;
Empty set (0.07 sec)
mysql>set profiling=1;
Query OK, 0 rows affected (0.01 sec)

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

…tlogs,db,table and add profiling table in order to Compatible with mysql (apache#22702)
add information_schema.metadata_name_idsfor quickly get catlogs,db,table.
1. table struct :
```mysql
mysql> desc internal.information_schema.metadata_name_ids;
+---------------+--------------+------+-------+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------------+--------------+------+-------+---------+-------+
| CATALOG_ID | BIGINT | Yes | false | NULL | |
| CATALOG_NAME | VARCHAR(512) | Yes | false | NULL | |
| DATABASE_ID | BIGINT | Yes | false | NULL | |
| DATABASE_NAME | VARCHAR(64) | Yes | false | NULL | |
| TABLE_ID | BIGINT | Yes | false | NULL | |
| TABLE_NAME | VARCHAR(64) | Yes | false | NULL | |
+---------------+--------------+------+-------+---------+-------+
6 rows in set (0.00 sec)
mysql> select * from internal.information_schema.metadata_name_ids where CATALOG_NAME="hive1" limit 1 \G;
*************************** 1. row ***************************
CATALOG_ID: 113008
CATALOG_NAME: hive1
DATABASE_ID: 113042
DATABASE_NAME: ssb1_parquet
TABLE_ID: 114009
TABLE_NAME: dates
1 row in set (0.07 sec)
```
2. when you create / drop catalog , need not refresh catalog .
```mysql
mysql> select count(*) from internal.information_schema.metadata_name_ids\G;
*************************** 1. row ***************************
count(*): 21301
1 row in set (0.34 sec)
mysql> drop catalog hive2;
Query OK, 0 rows affected (0.01 sec)
mysql> select count(*) from internal.information_schema.metadata_name_ids\G;
*************************** 1. row ***************************
count(*): 10665
1 row in set (0.04 sec)
mysql> create catalog hive3 ...
mysql> select count(*) from internal.information_schema.metadata_name_ids\G;
*************************** 1. row ***************************
count(*): 21301
1 row in set (0.32 sec)
```
3. create / drop table , need not refresh catalog .
```mysql
mysql> CREATE TABLE IF NOT EXISTS demo.example_tbl ... ;
mysql> select count(*) from internal.information_schema.metadata_name_ids\G;
*************************** 1. row ***************************
count(*): 10666
1 row in set (0.04 sec)
mysql> drop table demo.example_tbl;
Query OK, 0 rows affected (0.01 sec)
mysql> select count(*) from internal.information_schema.metadata_name_ids\G;
*************************** 1. row ***************************
count(*): 10665
1 row in set (0.04 sec)
```
4. you can set query time , prevent queries from taking too long .
```
fe.conf : query_metadata_name_ids_timeout
the time used to obtain all tables in one database
```
5. add information_schema.profiling in order to Compatible with mysql
```mysql
mysql> select * from information_schema.profiling;
Empty set (0.07 sec)
mysql> set profiling=1;
Query OK, 0 rows affected (0.01 sec)
```
@github-actionsgithub-actionsBot added area/planner Issues or PRs related to the query planner kind/test labels Sep 6, 2023
@github-actions

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@xiaokang
xiaokang merged commit 01ee595 into apache:branch-2.0Sep 6, 2023
@github-actions

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

xiaokang added a commit that referenced this pull request Sep 6, 2023
…y get catlogs,db,table and add profiling table in order to Compatible with mysql (#22702) (#23980)"
This reverts commit 01ee595.
xiaokang added a commit that referenced this pull request Sep 6, 2023
…y get catlogs,db,table and add profiling table in order to Compatible with mysql (#22702) (#23980)" (#24002)
This reverts commit 01ee595.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/plannerIssues or PRs related to the query plannerdev/2.0.2-mergedkind/test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@hubgeter@xiaokang