ARROW-12108: [Rust] [DataFusion] Implement SHOW TABLES by alamb · Pull Request #9847 · apache/arrow

alamb · 2021-03-30T14:45:54Z

Rationale

Accessing the list of tables via select * from information_schema.tables (introduced in #9818) is a lot to type

See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit#

Proposal

Add support for SHOW TABLES command.

Commentary

This is different than both postgres (which uses \d in psql for this purpose), and MySQL (which uses DESCRIBE).

I could be convinced that we should not add SHOW TABLES at all (and just stay with select * from information_schema.tables but I wanted to add the proposal)

Example Use

Setup:

echo "1,Foo,44.9" > /tmp/table.csv
echo "2,Bar,22.1" >> /tmp/table.csv
cargo run --bin datafusion-cli

Then run :

CREATE EXTERNAL TABLE t(a int, b varchar, c float)
STORED AS CSV
LOCATION '/tmp/table.csv';
> show tables;
+---------------+--------------------+------------+------------+
| table_catalog | table_schema | table_name | table_type |
+---------------+--------------------+------------+------------+
| datafusion | public | t | BASE TABLE |
| datafusion | information_schema | tables | VIEW |
+---------------+--------------------+------------+------------+
2 row in set. Query took 0 seconds.

github-actions · 2021-03-31T00:24:35Z

https://issues.apache.org/jira/browse/ARROW-12108

alamb · 2021-03-31T12:16:34Z

FYI @Dandandan @returnString @andygrove -- what do you think about adding support for SHOW TABLES?

andygrove

This is great. Thanks @alamb

jorgecarleitao

! Nice and sweet. Also, great coverage 👍

returnString · 2021-03-31T14:20:19Z

Great stuff - I think it's super useful as an accessible way to view the catalog structure outside of GUI tooling :)

alamb · 2021-03-31T17:36:42Z

Thanks all!

# Rationale Accessing the list of columns via `select * from information_schema.columns` (introduced in #9840) is a lot to type See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit# This is a sister PR to `SHOW TABLES` here: #9847 # Proposal Add support for `SHOW COLUMNS FROM <table>` command. Following the MySQL syntax supported by sqlparser: https://dev.mysql.com/doc/refman/8.0/en/show-columns.html # Example Use Setup: ``` echo "1,Foo,44.9" > /tmp/table.csv echo "2,Bar,22.1" >> /tmp/table.csv cargo run --bin datafusion-cli ``` Then run : ``` > CREATE EXTERNAL TABLE t(a int, b varchar, c float) STORED AS CSV LOCATION '/tmp/table.csv'; 0 rows in set. Query took 0 seconds. > show columns from t; +---------------+--------------+------------+-------------+-----------+-------------+ | table_catalog | table_schema | table_name | column_name | data_type | is_nullable | +---------------+--------------+------------+-------------+-----------+-------------+ | datafusion | public | t | a | Int32 | NO | | datafusion | public | t | b | Utf8 | NO | | datafusion | public | t | c | Float32 | NO | +---------------+--------------+------------+-------------+-----------+-------------+ 3 row in set. Query took 0 seconds. ``` # Commentary Note that the identifiers are case sensitive (which is a more general problem that affects all name resolution, not just `SHOW COLUMNS`). Ideally this should also work: ``` > show columns from T; Plan("Unknown relation for SHOW COLUMNS: T") > select * from T; Plan("Table or CTE with name \'T\' not found") ``` Closes#9866 from alamb/alamb/show_columns Authored-by: Andrew Lamb <andrew@nerdnetworks.org> Signed-off-by: Andrew Lamb <andrew@nerdnetworks.org>

alamb · 2021-03-30T14:45:54Z

Rationale

Accessing the list of tables via select * from information_schema.tables (introduced in #9818) is a lot to type

See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit#

Proposal

Add support for SHOW TABLES command.

Commentary

This is different than both postgres (which uses \d in psql for this purpose), and MySQL (which uses DESCRIBE).

I could be convinced that we should not add SHOW TABLES at all (and just stay with select * from information_schema.tables but I wanted to add the proposal)

Example Use

Setup:

echo "1,Foo,44.9" > /tmp/table.csv
echo "2,Bar,22.1" >> /tmp/table.csv
cargo run --bin datafusion-cli

Then run :

CREATE EXTERNAL TABLE t(a int, b varchar, c float)
STORED AS CSV
LOCATION '/tmp/table.csv';
> show tables;
+---------------+--------------------+------------+------------+
| table_catalog | table_schema | table_name | table_type |
+---------------+--------------------+------------+------------+
| datafusion | public | t | BASE TABLE |
| datafusion | information_schema | tables | VIEW |
+---------------+--------------------+------------+------------+
2 row in set. Query took 0 seconds.

github-actions · 2021-03-31T00:24:35Z

https://issues.apache.org/jira/browse/ARROW-12108

alamb · 2021-03-31T12:16:34Z

FYI @Dandandan @returnString @andygrove -- what do you think about adding support for SHOW TABLES?

andygrove

This is great. Thanks @alamb

jorgecarleitao

! Nice and sweet. Also, great coverage 👍

returnString · 2021-03-31T14:20:19Z

Great stuff - I think it's super useful as an accessible way to view the catalog structure outside of GUI tooling :)

alamb · 2021-03-31T17:36:42Z

Thanks all!

# Rationale Accessing the list of columns via `select * from information_schema.columns` (introduced in #9840) is a lot to type See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit# This is a sister PR to `SHOW TABLES` here: #9847 # Proposal Add support for `SHOW COLUMNS FROM <table>` command. Following the MySQL syntax supported by sqlparser: https://dev.mysql.com/doc/refman/8.0/en/show-columns.html # Example Use Setup: ``` echo "1,Foo,44.9" > /tmp/table.csv echo "2,Bar,22.1" >> /tmp/table.csv cargo run --bin datafusion-cli ``` Then run : ``` > CREATE EXTERNAL TABLE t(a int, b varchar, c float) STORED AS CSV LOCATION '/tmp/table.csv'; 0 rows in set. Query took 0 seconds. > show columns from t; +---------------+--------------+------------+-------------+-----------+-------------+ | table_catalog | table_schema | table_name | column_name | data_type | is_nullable | +---------------+--------------+------------+-------------+-----------+-------------+ | datafusion | public | t | a | Int32 | NO | | datafusion | public | t | b | Utf8 | NO | | datafusion | public | t | c | Float32 | NO | +---------------+--------------+------------+-------------+-----------+-------------+ 3 row in set. Query took 0 seconds. ``` # Commentary Note that the identifiers are case sensitive (which is a more general problem that affects all name resolution, not just `SHOW COLUMNS`). Ideally this should also work: ``` > show columns from T; Plan("Unknown relation for SHOW COLUMNS: T") > select * from T; Plan("Table or CTE with name \'T\' not found") ``` Closes#9866 from alamb/alamb/show_columns Authored-by: Andrew Lamb <andrew@nerdnetworks.org> Signed-off-by: Andrew Lamb <andrew@nerdnetworks.org>

alamb · 2021-03-30T14:45:54Z

Rationale

Accessing the list of tables via select * from information_schema.tables (introduced in #9818) is a lot to type

See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit#

Proposal

Add support for SHOW TABLES command.

Commentary

This is different than both postgres (which uses \d in psql for this purpose), and MySQL (which uses DESCRIBE).

I could be convinced that we should not add SHOW TABLES at all (and just stay with select * from information_schema.tables but I wanted to add the proposal)

Example Use

Setup:

echo "1,Foo,44.9" > /tmp/table.csv
echo "2,Bar,22.1" >> /tmp/table.csv
cargo run --bin datafusion-cli

Then run :

CREATE EXTERNAL TABLE t(a int, b varchar, c float)
STORED AS CSV
LOCATION '/tmp/table.csv';
> show tables;
+---------------+--------------------+------------+------------+
| table_catalog | table_schema | table_name | table_type |
+---------------+--------------------+------------+------------+
| datafusion | public | t | BASE TABLE |
| datafusion | information_schema | tables | VIEW |
+---------------+--------------------+------------+------------+
2 row in set. Query took 0 seconds.

github-actions · 2021-03-31T00:24:35Z

https://issues.apache.org/jira/browse/ARROW-12108

alamb · 2021-03-31T12:16:34Z

FYI @Dandandan @returnString @andygrove -- what do you think about adding support for SHOW TABLES?

andygrove

This is great. Thanks @alamb

jorgecarleitao

! Nice and sweet. Also, great coverage 👍

returnString · 2021-03-31T14:20:19Z

Great stuff - I think it's super useful as an accessible way to view the catalog structure outside of GUI tooling :)

alamb · 2021-03-31T17:36:42Z

Thanks all!

# Rationale Accessing the list of columns via `select * from information_schema.columns` (introduced in #9840) is a lot to type See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit# This is a sister PR to `SHOW TABLES` here: #9847 # Proposal Add support for `SHOW COLUMNS FROM <table>` command. Following the MySQL syntax supported by sqlparser: https://dev.mysql.com/doc/refman/8.0/en/show-columns.html # Example Use Setup: ``` echo "1,Foo,44.9" > /tmp/table.csv echo "2,Bar,22.1" >> /tmp/table.csv cargo run --bin datafusion-cli ``` Then run : ``` > CREATE EXTERNAL TABLE t(a int, b varchar, c float) STORED AS CSV LOCATION '/tmp/table.csv'; 0 rows in set. Query took 0 seconds. > show columns from t; +---------------+--------------+------------+-------------+-----------+-------------+ | table_catalog | table_schema | table_name | column_name | data_type | is_nullable | +---------------+--------------+------------+-------------+-----------+-------------+ | datafusion | public | t | a | Int32 | NO | | datafusion | public | t | b | Utf8 | NO | | datafusion | public | t | c | Float32 | NO | +---------------+--------------+------------+-------------+-----------+-------------+ 3 row in set. Query took 0 seconds. ``` # Commentary Note that the identifiers are case sensitive (which is a more general problem that affects all name resolution, not just `SHOW COLUMNS`). Ideally this should also work: ``` > show columns from T; Plan("Unknown relation for SHOW COLUMNS: T") > select * from T; Plan("Table or CTE with name \'T\' not found") ``` Closes#9866 from alamb/alamb/show_columns Authored-by: Andrew Lamb <andrew@nerdnetworks.org> Signed-off-by: Andrew Lamb <andrew@nerdnetworks.org>

alamb · 2021-03-30T14:45:54Z

Rationale

Accessing the list of tables via select * from information_schema.tables (introduced in #9818) is a lot to type

See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit#

Proposal

Add support for SHOW TABLES command.

Commentary

This is different than both postgres (which uses \d in psql for this purpose), and MySQL (which uses DESCRIBE).

I could be convinced that we should not add SHOW TABLES at all (and just stay with select * from information_schema.tables but I wanted to add the proposal)

Example Use

Setup:

echo "1,Foo,44.9" > /tmp/table.csv
echo "2,Bar,22.1" >> /tmp/table.csv
cargo run --bin datafusion-cli

Then run :

CREATE EXTERNAL TABLE t(a int, b varchar, c float)
STORED AS CSV
LOCATION '/tmp/table.csv';
> show tables;
+---------------+--------------------+------------+------------+
| table_catalog | table_schema | table_name | table_type |
+---------------+--------------------+------------+------------+
| datafusion | public | t | BASE TABLE |
| datafusion | information_schema | tables | VIEW |
+---------------+--------------------+------------+------------+
2 row in set. Query took 0 seconds.

github-actions · 2021-03-31T00:24:35Z

https://issues.apache.org/jira/browse/ARROW-12108

alamb · 2021-03-31T12:16:34Z

FYI @Dandandan @returnString @andygrove -- what do you think about adding support for SHOW TABLES?

andygrove

This is great. Thanks @alamb

jorgecarleitao

! Nice and sweet. Also, great coverage 👍

returnString · 2021-03-31T14:20:19Z

Great stuff - I think it's super useful as an accessible way to view the catalog structure outside of GUI tooling :)

alamb · 2021-03-31T17:36:42Z

Thanks all!

# Rationale Accessing the list of columns via `select * from information_schema.columns` (introduced in #9840) is a lot to type See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit# This is a sister PR to `SHOW TABLES` here: #9847 # Proposal Add support for `SHOW COLUMNS FROM <table>` command. Following the MySQL syntax supported by sqlparser: https://dev.mysql.com/doc/refman/8.0/en/show-columns.html # Example Use Setup: ``` echo "1,Foo,44.9" > /tmp/table.csv echo "2,Bar,22.1" >> /tmp/table.csv cargo run --bin datafusion-cli ``` Then run : ``` > CREATE EXTERNAL TABLE t(a int, b varchar, c float) STORED AS CSV LOCATION '/tmp/table.csv'; 0 rows in set. Query took 0 seconds. > show columns from t; +---------------+--------------+------------+-------------+-----------+-------------+ | table_catalog | table_schema | table_name | column_name | data_type | is_nullable | +---------------+--------------+------------+-------------+-----------+-------------+ | datafusion | public | t | a | Int32 | NO | | datafusion | public | t | b | Utf8 | NO | | datafusion | public | t | c | Float32 | NO | +---------------+--------------+------------+-------------+-----------+-------------+ 3 row in set. Query took 0 seconds. ``` # Commentary Note that the identifiers are case sensitive (which is a more general problem that affects all name resolution, not just `SHOW COLUMNS`). Ideally this should also work: ``` > show columns from T; Plan("Unknown relation for SHOW COLUMNS: T") > select * from T; Plan("Table or CTE with name \'T\' not found") ``` Closes#9866 from alamb/alamb/show_columns Authored-by: Andrew Lamb <andrew@nerdnetworks.org> Signed-off-by: Andrew Lamb <andrew@nerdnetworks.org>

alamb · 2021-03-30T14:45:54Z

Rationale

Accessing the list of tables via select * from information_schema.tables (introduced in #9818) is a lot to type

See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit#

Proposal

Add support for SHOW TABLES command.

Commentary

This is different than both postgres (which uses \d in psql for this purpose), and MySQL (which uses DESCRIBE).

I could be convinced that we should not add SHOW TABLES at all (and just stay with select * from information_schema.tables but I wanted to add the proposal)

Example Use

Setup:

echo "1,Foo,44.9" > /tmp/table.csv
echo "2,Bar,22.1" >> /tmp/table.csv
cargo run --bin datafusion-cli

Then run :

CREATE EXTERNAL TABLE t(a int, b varchar, c float)
STORED AS CSV
LOCATION '/tmp/table.csv';
> show tables;
+---------------+--------------------+------------+------------+
| table_catalog | table_schema | table_name | table_type |
+---------------+--------------------+------------+------------+
| datafusion | public | t | BASE TABLE |
| datafusion | information_schema | tables | VIEW |
+---------------+--------------------+------------+------------+
2 row in set. Query took 0 seconds.

github-actions · 2021-03-31T00:24:35Z

https://issues.apache.org/jira/browse/ARROW-12108

alamb · 2021-03-31T12:16:34Z

FYI @Dandandan @returnString @andygrove -- what do you think about adding support for SHOW TABLES?

andygrove

This is great. Thanks @alamb

jorgecarleitao

! Nice and sweet. Also, great coverage 👍

returnString · 2021-03-31T14:20:19Z

Great stuff - I think it's super useful as an accessible way to view the catalog structure outside of GUI tooling :)

alamb · 2021-03-31T17:36:42Z

Thanks all!

# Rationale Accessing the list of columns via `select * from information_schema.columns` (introduced in #9840) is a lot to type See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit# This is a sister PR to `SHOW TABLES` here: #9847 # Proposal Add support for `SHOW COLUMNS FROM <table>` command. Following the MySQL syntax supported by sqlparser: https://dev.mysql.com/doc/refman/8.0/en/show-columns.html # Example Use Setup: ``` echo "1,Foo,44.9" > /tmp/table.csv echo "2,Bar,22.1" >> /tmp/table.csv cargo run --bin datafusion-cli ``` Then run : ``` > CREATE EXTERNAL TABLE t(a int, b varchar, c float) STORED AS CSV LOCATION '/tmp/table.csv'; 0 rows in set. Query took 0 seconds. > show columns from t; +---------------+--------------+------------+-------------+-----------+-------------+ | table_catalog | table_schema | table_name | column_name | data_type | is_nullable | +---------------+--------------+------------+-------------+-----------+-------------+ | datafusion | public | t | a | Int32 | NO | | datafusion | public | t | b | Utf8 | NO | | datafusion | public | t | c | Float32 | NO | +---------------+--------------+------------+-------------+-----------+-------------+ 3 row in set. Query took 0 seconds. ``` # Commentary Note that the identifiers are case sensitive (which is a more general problem that affects all name resolution, not just `SHOW COLUMNS`). Ideally this should also work: ``` > show columns from T; Plan("Unknown relation for SHOW COLUMNS: T") > select * from T; Plan("Table or CTE with name \'T\' not found") ``` Closes#9866 from alamb/alamb/show_columns Authored-by: Andrew Lamb <andrew@nerdnetworks.org> Signed-off-by: Andrew Lamb <andrew@nerdnetworks.org>

alamb · 2021-03-30T14:45:54Z

Rationale

Accessing the list of tables via select * from information_schema.tables (introduced in #9818) is a lot to type

See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit#

Proposal

Add support for SHOW TABLES command.

Commentary

This is different than both postgres (which uses \d in psql for this purpose), and MySQL (which uses DESCRIBE).

I could be convinced that we should not add SHOW TABLES at all (and just stay with select * from information_schema.tables but I wanted to add the proposal)

Example Use

Setup:

echo "1,Foo,44.9" > /tmp/table.csv
echo "2,Bar,22.1" >> /tmp/table.csv
cargo run --bin datafusion-cli

Then run :

CREATE EXTERNAL TABLE t(a int, b varchar, c float)
STORED AS CSV
LOCATION '/tmp/table.csv';
> show tables;
+---------------+--------------------+------------+------------+
| table_catalog | table_schema | table_name | table_type |
+---------------+--------------------+------------+------------+
| datafusion | public | t | BASE TABLE |
| datafusion | information_schema | tables | VIEW |
+---------------+--------------------+------------+------------+
2 row in set. Query took 0 seconds.

github-actions · 2021-03-31T00:24:35Z

https://issues.apache.org/jira/browse/ARROW-12108

alamb · 2021-03-31T12:16:34Z

FYI @Dandandan @returnString @andygrove -- what do you think about adding support for SHOW TABLES?

andygrove

This is great. Thanks @alamb

jorgecarleitao

! Nice and sweet. Also, great coverage 👍

returnString · 2021-03-31T14:20:19Z

Great stuff - I think it's super useful as an accessible way to view the catalog structure outside of GUI tooling :)

alamb · 2021-03-31T17:36:42Z

Thanks all!

# Rationale Accessing the list of columns via `select * from information_schema.columns` (introduced in #9840) is a lot to type See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit# This is a sister PR to `SHOW TABLES` here: #9847 # Proposal Add support for `SHOW COLUMNS FROM <table>` command. Following the MySQL syntax supported by sqlparser: https://dev.mysql.com/doc/refman/8.0/en/show-columns.html # Example Use Setup: ``` echo "1,Foo,44.9" > /tmp/table.csv echo "2,Bar,22.1" >> /tmp/table.csv cargo run --bin datafusion-cli ``` Then run : ``` > CREATE EXTERNAL TABLE t(a int, b varchar, c float) STORED AS CSV LOCATION '/tmp/table.csv'; 0 rows in set. Query took 0 seconds. > show columns from t; +---------------+--------------+------------+-------------+-----------+-------------+ | table_catalog | table_schema | table_name | column_name | data_type | is_nullable | +---------------+--------------+------------+-------------+-----------+-------------+ | datafusion | public | t | a | Int32 | NO | | datafusion | public | t | b | Utf8 | NO | | datafusion | public | t | c | Float32 | NO | +---------------+--------------+------------+-------------+-----------+-------------+ 3 row in set. Query took 0 seconds. ``` # Commentary Note that the identifiers are case sensitive (which is a more general problem that affects all name resolution, not just `SHOW COLUMNS`). Ideally this should also work: ``` > show columns from T; Plan("Unknown relation for SHOW COLUMNS: T") > select * from T; Plan("Table or CTE with name \'T\' not found") ``` Closes#9866 from alamb/alamb/show_columns Authored-by: Andrew Lamb <andrew@nerdnetworks.org> Signed-off-by: Andrew Lamb <andrew@nerdnetworks.org>

alamb · 2021-03-30T14:45:54Z

Rationale

Accessing the list of tables via select * from information_schema.tables (introduced in #9818) is a lot to type

See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit#

Proposal

Add support for SHOW TABLES command.

Commentary

This is different than both postgres (which uses \d in psql for this purpose), and MySQL (which uses DESCRIBE).

I could be convinced that we should not add SHOW TABLES at all (and just stay with select * from information_schema.tables but I wanted to add the proposal)

Example Use

Setup:

echo "1,Foo,44.9" > /tmp/table.csv
echo "2,Bar,22.1" >> /tmp/table.csv
cargo run --bin datafusion-cli

Then run :

CREATE EXTERNAL TABLE t(a int, b varchar, c float)
STORED AS CSV
LOCATION '/tmp/table.csv';
> show tables;
+---------------+--------------------+------------+------------+
| table_catalog | table_schema | table_name | table_type |
+---------------+--------------------+------------+------------+
| datafusion | public | t | BASE TABLE |
| datafusion | information_schema | tables | VIEW |
+---------------+--------------------+------------+------------+
2 row in set. Query took 0 seconds.

github-actions · 2021-03-31T00:24:35Z

https://issues.apache.org/jira/browse/ARROW-12108

alamb · 2021-03-31T12:16:34Z

FYI @Dandandan @returnString @andygrove -- what do you think about adding support for SHOW TABLES?

andygrove

This is great. Thanks @alamb

jorgecarleitao

! Nice and sweet. Also, great coverage 👍

returnString · 2021-03-31T14:20:19Z

Great stuff - I think it's super useful as an accessible way to view the catalog structure outside of GUI tooling :)

alamb · 2021-03-31T17:36:42Z

Thanks all!

# Rationale Accessing the list of columns via `select * from information_schema.columns` (introduced in #9840) is a lot to type See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit# This is a sister PR to `SHOW TABLES` here: #9847 # Proposal Add support for `SHOW COLUMNS FROM <table>` command. Following the MySQL syntax supported by sqlparser: https://dev.mysql.com/doc/refman/8.0/en/show-columns.html # Example Use Setup: ``` echo "1,Foo,44.9" > /tmp/table.csv echo "2,Bar,22.1" >> /tmp/table.csv cargo run --bin datafusion-cli ``` Then run : ``` > CREATE EXTERNAL TABLE t(a int, b varchar, c float) STORED AS CSV LOCATION '/tmp/table.csv'; 0 rows in set. Query took 0 seconds. > show columns from t; +---------------+--------------+------------+-------------+-----------+-------------+ | table_catalog | table_schema | table_name | column_name | data_type | is_nullable | +---------------+--------------+------------+-------------+-----------+-------------+ | datafusion | public | t | a | Int32 | NO | | datafusion | public | t | b | Utf8 | NO | | datafusion | public | t | c | Float32 | NO | +---------------+--------------+------------+-------------+-----------+-------------+ 3 row in set. Query took 0 seconds. ``` # Commentary Note that the identifiers are case sensitive (which is a more general problem that affects all name resolution, not just `SHOW COLUMNS`). Ideally this should also work: ``` > show columns from T; Plan("Unknown relation for SHOW COLUMNS: T") > select * from T; Plan("Table or CTE with name \'T\' not found") ``` Closes#9866 from alamb/alamb/show_columns Authored-by: Andrew Lamb <andrew@nerdnetworks.org> Signed-off-by: Andrew Lamb <andrew@nerdnetworks.org>

alamb · 2021-03-30T14:45:54Z

Rationale

Accessing the list of tables via select * from information_schema.tables (introduced in #9818) is a lot to type

See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit#

Proposal

Add support for SHOW TABLES command.

Commentary

This is different than both postgres (which uses \d in psql for this purpose), and MySQL (which uses DESCRIBE).

I could be convinced that we should not add SHOW TABLES at all (and just stay with select * from information_schema.tables but I wanted to add the proposal)

Example Use

Setup:

echo "1,Foo,44.9" > /tmp/table.csv
echo "2,Bar,22.1" >> /tmp/table.csv
cargo run --bin datafusion-cli

Then run :

CREATE EXTERNAL TABLE t(a int, b varchar, c float)
STORED AS CSV
LOCATION '/tmp/table.csv';
> show tables;
+---------------+--------------------+------------+------------+
| table_catalog | table_schema | table_name | table_type |
+---------------+--------------------+------------+------------+
| datafusion | public | t | BASE TABLE |
| datafusion | information_schema | tables | VIEW |
+---------------+--------------------+------------+------------+
2 row in set. Query took 0 seconds.

github-actions · 2021-03-31T00:24:35Z

https://issues.apache.org/jira/browse/ARROW-12108

alamb · 2021-03-31T12:16:34Z

FYI @Dandandan @returnString @andygrove -- what do you think about adding support for SHOW TABLES?

andygrove

This is great. Thanks @alamb

jorgecarleitao

! Nice and sweet. Also, great coverage 👍

returnString · 2021-03-31T14:20:19Z

Great stuff - I think it's super useful as an accessible way to view the catalog structure outside of GUI tooling :)

alamb · 2021-03-31T17:36:42Z

Thanks all!

# Rationale Accessing the list of columns via `select * from information_schema.columns` (introduced in #9840) is a lot to type See the doc for background: https://docs.google.com/document/d/12cpZUSNPqVH9Z0BBx6O8REu7TFqL-NPPAYCUPpDls1k/edit# This is a sister PR to `SHOW TABLES` here: #9847 # Proposal Add support for `SHOW COLUMNS FROM <table>` command. Following the MySQL syntax supported by sqlparser: https://dev.mysql.com/doc/refman/8.0/en/show-columns.html # Example Use Setup: ``` echo "1,Foo,44.9" > /tmp/table.csv echo "2,Bar,22.1" >> /tmp/table.csv cargo run --bin datafusion-cli ``` Then run : ``` > CREATE EXTERNAL TABLE t(a int, b varchar, c float) STORED AS CSV LOCATION '/tmp/table.csv'; 0 rows in set. Query took 0 seconds. > show columns from t; +---------------+--------------+------------+-------------+-----------+-------------+ | table_catalog | table_schema | table_name | column_name | data_type | is_nullable | +---------------+--------------+------------+-------------+-----------+-------------+ | datafusion | public | t | a | Int32 | NO | | datafusion | public | t | b | Utf8 | NO | | datafusion | public | t | c | Float32 | NO | +---------------+--------------+------------+-------------+-----------+-------------+ 3 row in set. Query took 0 seconds. ``` # Commentary Note that the identifiers are case sensitive (which is a more general problem that affects all name resolution, not just `SHOW COLUMNS`). Ideally this should also work: ``` > show columns from T; Plan("Unknown relation for SHOW COLUMNS: T") > select * from T; Plan("Table or CTE with name \'T\' not found") ``` Closes#9866 from alamb/alamb/show_columns Authored-by: Andrew Lamb <andrew@nerdnetworks.org> Signed-off-by: Andrew Lamb <andrew@nerdnetworks.org>

ARROW-12108: Implement SHOW TABLES
f29f726

nevi-me changed the title ~~ARROW-12108: Implement SHOW TABLES~~ARROW-12108: [Rust] [DataFusion] Implement SHOW TABLESMar 30, 2021

github-actionsBot added Component: Rust - DataFusion Component: Rust labels Mar 31, 2021

alamb requested review from andygrove and jorgecarleitao March 31, 2021 12:16

andygrove approved these changes Mar 31, 2021
View reviewed changes

jorgecarleitao approved these changes Mar 31, 2021
View reviewed changes

returnString approved these changes Mar 31, 2021
View reviewed changes

alamb closed this in 9a37f35Mar 31, 2021

alamb mentioned this pull request Mar 31, 2021
ARROW-12109: [Rust][DataFusion] Implement SHOW COLUMNS #9866
Closed

asfimport mentioned this pull request Mar 31, 2021
[Rust][DataFusion] Support SHOW TABLES#27931
Closed

ARROW-12108: Implement SHOW TABLES
f29f726

nevi-me changed the title ~~ARROW-12108: Implement SHOW TABLES~~ARROW-12108: [Rust] [DataFusion] Implement SHOW TABLESMar 30, 2021

github-actionsBot added Component: Rust - DataFusion Component: Rust labels Mar 31, 2021

alamb requested review from andygrove and jorgecarleitao March 31, 2021 12:16

andygrove approved these changes Mar 31, 2021
View reviewed changes

jorgecarleitao approved these changes Mar 31, 2021
View reviewed changes

returnString approved these changes Mar 31, 2021
View reviewed changes

alamb closed this in 9a37f35Mar 31, 2021

alamb mentioned this pull request Mar 31, 2021
ARROW-12109: [Rust][DataFusion] Implement SHOW COLUMNS #9866
Closed

asfimport mentioned this pull request Mar 31, 2021
[Rust][DataFusion] Support SHOW TABLES#27931
Closed

ARROW-12108: Implement SHOW TABLES
f29f726

nevi-me changed the title ~~ARROW-12108: Implement SHOW TABLES~~ARROW-12108: [Rust] [DataFusion] Implement SHOW TABLESMar 30, 2021

github-actionsBot added Component: Rust - DataFusion Component: Rust labels Mar 31, 2021

alamb requested review from andygrove and jorgecarleitao March 31, 2021 12:16

andygrove approved these changes Mar 31, 2021
View reviewed changes

jorgecarleitao approved these changes Mar 31, 2021
View reviewed changes

returnString approved these changes Mar 31, 2021
View reviewed changes

alamb closed this in 9a37f35Mar 31, 2021

alamb mentioned this pull request Mar 31, 2021
ARROW-12109: [Rust][DataFusion] Implement SHOW COLUMNS #9866
Closed

asfimport mentioned this pull request Mar 31, 2021
[Rust][DataFusion] Support SHOW TABLES#27931
Closed

ARROW-12108: Implement SHOW TABLES
f29f726

nevi-me changed the title ~~ARROW-12108: Implement SHOW TABLES~~ARROW-12108: [Rust] [DataFusion] Implement SHOW TABLESMar 30, 2021

github-actionsBot added Component: Rust - DataFusion Component: Rust labels Mar 31, 2021

alamb requested review from andygrove and jorgecarleitao March 31, 2021 12:16

andygrove approved these changes Mar 31, 2021
View reviewed changes

jorgecarleitao approved these changes Mar 31, 2021
View reviewed changes

returnString approved these changes Mar 31, 2021
View reviewed changes

alamb closed this in 9a37f35Mar 31, 2021

alamb mentioned this pull request Mar 31, 2021
ARROW-12109: [Rust][DataFusion] Implement SHOW COLUMNS #9866
Closed

asfimport mentioned this pull request Mar 31, 2021
[Rust][DataFusion] Support SHOW TABLES#27931
Closed

ARROW-12108: Implement SHOW TABLES
f29f726

nevi-me changed the title ~~ARROW-12108: Implement SHOW TABLES~~ARROW-12108: [Rust] [DataFusion] Implement SHOW TABLESMar 30, 2021

github-actionsBot added Component: Rust - DataFusion Component: Rust labels Mar 31, 2021

alamb requested review from andygrove and jorgecarleitao March 31, 2021 12:16

andygrove approved these changes Mar 31, 2021
View reviewed changes

jorgecarleitao approved these changes Mar 31, 2021
View reviewed changes

returnString approved these changes Mar 31, 2021
View reviewed changes

alamb closed this in 9a37f35Mar 31, 2021

alamb mentioned this pull request Mar 31, 2021
ARROW-12109: [Rust][DataFusion] Implement SHOW COLUMNS #9866
Closed

asfimport mentioned this pull request Mar 31, 2021
[Rust][DataFusion] Support SHOW TABLES#27931
Closed

ARROW-12108: Implement SHOW TABLES
f29f726

nevi-me changed the title ~~ARROW-12108: Implement SHOW TABLES~~ARROW-12108: [Rust] [DataFusion] Implement SHOW TABLESMar 30, 2021

github-actionsBot added Component: Rust - DataFusion Component: Rust labels Mar 31, 2021

alamb requested review from andygrove and jorgecarleitao March 31, 2021 12:16

andygrove approved these changes Mar 31, 2021
View reviewed changes

jorgecarleitao approved these changes Mar 31, 2021
View reviewed changes

returnString approved these changes Mar 31, 2021
View reviewed changes

alamb closed this in 9a37f35Mar 31, 2021

alamb mentioned this pull request Mar 31, 2021
ARROW-12109: [Rust][DataFusion] Implement SHOW COLUMNS #9866
Closed

asfimport mentioned this pull request Mar 31, 2021
[Rust][DataFusion] Support SHOW TABLES#27931
Closed

ARROW-12108: Implement SHOW TABLES
f29f726

nevi-me changed the title ~~ARROW-12108: Implement SHOW TABLES~~ARROW-12108: [Rust] [DataFusion] Implement SHOW TABLESMar 30, 2021

github-actionsBot added Component: Rust - DataFusion Component: Rust labels Mar 31, 2021

alamb requested review from andygrove and jorgecarleitao March 31, 2021 12:16

andygrove approved these changes Mar 31, 2021
View reviewed changes

jorgecarleitao approved these changes Mar 31, 2021
View reviewed changes

returnString approved these changes Mar 31, 2021
View reviewed changes

alamb closed this in 9a37f35Mar 31, 2021

alamb mentioned this pull request Mar 31, 2021
ARROW-12109: [Rust][DataFusion] Implement SHOW COLUMNS #9866
Closed

asfimport mentioned this pull request Mar 31, 2021
[Rust][DataFusion] Support SHOW TABLES#27931
Closed

ARROW-12108: Implement SHOW TABLES
f29f726

nevi-me changed the title ~~ARROW-12108: Implement SHOW TABLES~~ARROW-12108: [Rust] [DataFusion] Implement SHOW TABLESMar 30, 2021

github-actionsBot added Component: Rust - DataFusion Component: Rust labels Mar 31, 2021

alamb requested review from andygrove and jorgecarleitao March 31, 2021 12:16

andygrove approved these changes Mar 31, 2021
View reviewed changes

jorgecarleitao approved these changes Mar 31, 2021
View reviewed changes

returnString approved these changes Mar 31, 2021
View reviewed changes

alamb closed this in 9a37f35Mar 31, 2021

alamb mentioned this pull request Mar 31, 2021
ARROW-12109: [Rust][DataFusion] Implement SHOW COLUMNS #9866
Closed

asfimport mentioned this pull request Mar 31, 2021
[Rust][DataFusion] Support SHOW TABLES#27931
Closed

Uh oh!

Conversation

alamb commented Mar 30, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale

Proposal

Commentary

Example Use

Uh oh!

github-actionsBot commented Mar 31, 2021

Uh oh!

alamb commented Mar 31, 2021

Uh oh!

andygrove left a comment

Choose a reason for hiding this comment

Uh oh!

jorgecarleitao left a comment

Choose a reason for hiding this comment

Uh oh!

returnString commented Mar 31, 2021

Uh oh!

alamb commented Mar 31, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

alamb commented Mar 30, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale

Proposal

Commentary

Example Use

Uh oh!

github-actionsBot commented Mar 31, 2021

Uh oh!

alamb commented Mar 31, 2021

Uh oh!

andygrove left a comment

Choose a reason for hiding this comment

Uh oh!

jorgecarleitao left a comment

Choose a reason for hiding this comment

Uh oh!

returnString commented Mar 31, 2021

Uh oh!

alamb commented Mar 31, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

alamb commented Mar 30, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale

Proposal

Commentary

Example Use

Uh oh!

github-actionsBot commented Mar 31, 2021

Uh oh!

alamb commented Mar 31, 2021

Uh oh!

andygrove left a comment

Choose a reason for hiding this comment

Uh oh!

jorgecarleitao left a comment

Choose a reason for hiding this comment

alamb commented Mar 30, 2021•
edited
Loading

alamb commented Mar 30, 2021•
edited
Loading

alamb commented Mar 30, 2021•
edited
Loading

alamb commented Mar 30, 2021•
edited
Loading

alamb commented Mar 30, 2021•
edited
Loading