Skip to content

[SPARK-14460] [SQL] properly handling of column name contains space - #12252

Closed
bomeng wants to merge 11 commits into
apache:masterfrom
bomeng:SPARK-14460
Closed

[SPARK-14460] [SQL] properly handling of column name contains space#12252
bomeng wants to merge 11 commits into
apache:masterfrom
bomeng:SPARK-14460

Conversation

@bomeng

@bomengbomeng commented Apr 8, 2016•
edited
Loading

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Although it is not recommended, table can be created with column name containing space. For example,

create table test.people1 (name TEXT(32) NOT NULL,the id INTEGER NOT NULL)

When we query the table via JDBC, we also need to quote the column name in the same way if the column name contains space.

I am leveraging the quote method in the dialect, it will add the proper quote to the column name based on its data source.

How was this patch tested?

I've updated the test case to test this scenario.

@SparkQA

SparkQA commented Apr 8, 2016

Copy link
Copy Markdown

Test build #55312 has finished for PR 12252 at commit 51b7f86.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Apr 8, 2016

Copy link
Copy Markdown

Test build #55309 has finished for PR 12252 at commit 88935c5.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Apr 8, 2016

Copy link
Copy Markdown

Test build #55330 has finished for PR 12252 at commit d0c86ea.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Apr 8, 2016

Copy link
Copy Markdown

Test build #55328 has finished for PR 12252 at commit 9ffb462.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@bomeng

bomeng commented Apr 11, 2016

Copy link
Copy Markdown
ContributorAuthor

@andrewor14 Could you please review my codes?

@SparkQA

SparkQA commented Apr 14, 2016

Copy link
Copy Markdown

Test build #55851 has finished for PR 12252 at commit ecd52bf.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Apr 14, 2016

Copy link
Copy Markdown

Test build #55854 has finished for PR 12252 at commit b70f21f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

ash211
ash211 reviewed Apr 15, 2016
View reviewed changes
Comment threadsql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala
*/
def schemaString(df: DataFrame, url: String): String = {
def schemaString(dialect: JdbcDialect, df: DataFrame, url: String): String = {
val sb = new StringBuilder()

@ash211ash211Apr 15, 2016

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the new dialect you're passing in different from this one in some way?

@bomengbomengApr 15, 2016

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose to pass in dialect is to get proper quote for columns based on its data source. Any suggestion?

@ash211ash211Apr 15, 2016

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You pass in a parameter named dialect to the schemaString method, but there's also the dialect that comes from JdbcDialects.get(url) --- that's the duplicate I was trying to point out

@bomengbomengApr 15, 2016

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing out. I've modified the codes. Please check it out.

@SparkQA

SparkQA commented Apr 15, 2016

Copy link
Copy Markdown

Test build #55889 has finished for PR 12252 at commit 715777a.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Apr 15, 2016

Copy link
Copy Markdown

Test build #55891 has finished for PR 12252 at commit 675700f.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Apr 20, 2016

Copy link
Copy Markdown

Test build #56384 has finished for PR 12252 at commit b151065.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan

cloud-fan commented May 25, 2017

Copy link
Copy Markdown
Contributor

seems it's fixed in #15662 ?

@HyukjinKwon

HyukjinKwon commented Jun 2, 2017

Copy link
Copy Markdown
Member

ping @bomeng for ^.

@HyukjinKwonHyukjinKwon mentioned this pull request Jun 7, 2017
Closed
@asfgitasfgit closed this in b771fedJun 8, 2017
zifeif2 pushed a commit to zifeif2/spark that referenced this pull request Nov 22, 2025
@HyukjinKwon@zifeif2
[INFRA] Close stale PRs
ce94f63
# What changes were proposed in this pull request?
This PR proposes to close stale PRs, mostly the same instances with apache#18017Closesapache#11459Closesapache#13833Closesapache#13720Closesapache#12506Closesapache#12456Closesapache#12252Closesapache#17689Closesapache#17791Closesapache#18163Closesapache#17640Closesapache#17926Closesapache#18163Closesapache#12506Closesapache#18044Closesapache#14036Closesapache#15831Closesapache#14461Closesapache#17638Closesapache#18222
Added:
Closesapache#18045Closesapache#18061Closesapache#18010Closesapache#18041Closesapache#18124Closesapache#18130Closesapache#12217
Added:
Closesapache#16291Closesapache#17480Closesapache#14995
Added:
Closesapache#12835Closesapache#17141
## How was this patch tested?
N/A
Author: hyukjinkwon <gurwls223@gmail.com>
Closesapache#18223 from HyukjinKwon/close-stale-prs.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@bomeng@SparkQA@cloud-fan@HyukjinKwon@ash211