[SPARK-17783] [SQL] [BACKPORT-2.0] Hide Credentials in CREATE and DESC FORMATTED/EXTENDED a PERSISTENT/TEMP Table for JDBC

[gatorsmile]

What changes were proposed in this pull request?

This PR is to backport #15358 to Spark 2.0.

---

We should never expose the Credentials in the EXPLAIN and DESC FORMATTED/EXTENDED command. However, below commands exposed the credentials.

In the related PR: #10452

URL patterns to specify credential seems to be vary between different databases.

Thus, we hide the whole url value if it contains the keyword password. We also hide the password property.

Before the fix, the command outputs look like:

CREATETABLEtab1
USING org.apache.spark.sql.jdbc
OPTIONS (
url 'jdbc:h2:mem:testdb0;user=testUser;password=testPass',
dbtable 'TEST.PEOPLE',
user 'testUser',
password '$password')
DESC FORMATTED tab1
DESC EXTENDED tab1

Before the fix,

• The output of SQL statement EXPLAIN

== Physical Plan ==
ExecutedCommand
+- CreateDataSourceTableCommand CatalogTable(
Table: `tab1`
Created: Wed Nov 16 23:00:10 PST 2016
Last Access: Wed Dec 31 15:59:59 PST 1969
Type: MANAGED
Provider: org.apache.spark.sql.jdbc
Storage(Properties: [url=jdbc:h2:mem:testdb0;user=testUser;password=testPass, dbtable=TEST.PEOPLE, user=testUser, password=testPass])), false

• The output of DESC FORMATTED

...
|Storage Desc Parameters: | | |
| url |jdbc:h2:mem:testdb0;user=testUser;password=testPass | |
| dbtable |TEST.PEOPLE | |
| user |testUser | |
| password |testPass | |
+----------------------------+------------------------------------------------------------------+-------+

• The output of DESC EXTENDED

|# Detailed Table Information|CatalogTable(
Table: `default`.`tab1`
Created: Wed Nov 16 23:00:10 PST 2016
Last Access: Wed Dec 31 15:59:59 PST 1969
Type: MANAGED
Schema: [StructField(NAME,StringType,false), StructField(THEID,IntegerType,false)]
Provider: org.apache.spark.sql.jdbc
Storage(Location: file:/Users/xiaoli/IdeaProjects/sparkDelivery/spark-warehouse/tab1, Properties: [url=jdbc:h2:mem:testdb0;user=testUser;password=testPass, dbtable=TEST.PEOPLE, user=testUser, password=testPass]))| |

After the fix,

• The output of SQL statement EXPLAIN

== Physical Plan ==
ExecutedCommand
+- CreateDataSourceTableCommand CatalogTable(
Table: `tab1`
Created: Wed Nov 16 22:43:49 PST 2016
Last Access: Wed Dec 31 15:59:59 PST 1969
Type: MANAGED
Provider: org.apache.spark.sql.jdbc
Storage(Properties: [url=###, dbtable=TEST.PEOPLE, user=testUser, password=###])), false

• The output of DESC FORMATTED

...
|Storage Desc Parameters: | | |
| url |### | |
| dbtable |TEST.PEOPLE | |
| user |testUser | |
| password |### | |
+----------------------------+------------------------------------------------------------------+-------+

• The output of DESC EXTENDED

|# Detailed Table Information|CatalogTable(
Table: `default`.`tab1`
Created: Wed Nov 16 22:43:49 PST 2016
Last Access: Wed Dec 31 15:59:59 PST 1969
Type: MANAGED
Schema: [StructField(NAME,StringType,false), StructField(THEID,IntegerType,false)]
Provider: org.apache.spark.sql.jdbc
Storage(Location: file:/Users/xiaoli/IdeaProjects/sparkDelivery/spark-warehouse/tab1, Properties: [url=###, dbtable=TEST.PEOPLE, user=testUser, password=###]))| |

How was this patch tested?

Added test cases

[gatorsmile]

Above is the extra changes we need to make, because Spark 2.1 we unified CREATE TABLE plan nodes.

[gatorsmile]

Above is another extra change we need to make, because Spark 2.1 we unified CREATE TABLE plan nodes.

[gatorsmile]

cc @hvanhovell

[SparkQA]

Test build #69271 has finished for PR 16047 at commit 105961b.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[hvanhovell]

LGTM - merging to 2.0. Thanks!

Can you close?