Skip to content

[SPARK-19952][SQL] Remove various analysis exceptions - #17716

Closed
hvanhovell wants to merge 2 commits into
apache:masterfrom
hvanhovell:SPARK-19952
Closed

[SPARK-19952][SQL] Remove various analysis exceptions#17716
hvanhovell wants to merge 2 commits into
apache:masterfrom
hvanhovell:SPARK-19952

Conversation

@hvanhovell

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

We currently have quite a few analysis exception subclasses, the problem with these is that they are not well supported throughout Spark (in pySpark for example) and that the added value is limited. This PR removes the no such object and the already exists exceptions, and replaces them by a factory methods that create analysis exceptions with the required error messages.

How was this patch tested?

Modified existing tests.

case _: NoSuchTableException =>
u.failAnalysis(s"Table or view not found: ${tableIdentWithDb.unquotedString}")
// If the database is defined and that database is not found, throw an AnalysisException.
if (!tableIdentWithDb.database.exists(catalog.databaseExists)) {

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should move this into catalog.lookupRelation and attach the location here.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about

try {
catalog.lookupRelation(tableIdentWithDb)
} catch {
case a: AnalysisException => u.failAnalysis(a.message)
}

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, could work. This change is also causing the tests to fail, so I am revisiting it.

@hvanhovell

Copy link
Copy Markdown
ContributorAuthor

cc @cloud-fan@ueshin@rxin

@SparkQA

Copy link
Copy Markdown

Test build #76031 has finished for PR 17716 at commit 6dc3204.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

Copy link
Copy Markdown

Test build #76039 has finished for PR 17716 at commit 79ee4a6.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.


private def assertNoSuchTable(query: String): Unit = {
intercept[NoSuchTableException] {
intercept[AnalysisException] {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AnalysisException is a pretty general Exception in Spark SQL. The future refactoring might introduce bugs without trigger any test case failure.

Could we check the error messages?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do

@HyukjinKwon

Copy link
Copy Markdown
Member

@hvanhovell, how is it going?

@HyukjinKwonHyukjinKwon mentioned this pull request Jun 7, 2017
@HyukjinKwonHyukjinKwon mentioned this pull request Jun 25, 2017
@jiangxb1987

Copy link
Copy Markdown
Contributor

ping @hvanhovell Are you still working on this?

zifeif2 pushed a commit to zifeif2/spark that referenced this pull request Nov 22, 2025
## What changes were proposed in this pull request?
This PR proposes to close stale PRs, mostly the same instances with apache#18017
I believe the author in apache#14807 removed his account.
Closesapache#7075Closesapache#8927Closesapache#9202Closesapache#9366Closesapache#10861Closesapache#11420Closesapache#12356Closesapache#13028Closesapache#13506Closesapache#14191Closesapache#14198Closesapache#14330Closesapache#14807Closesapache#15839Closesapache#16225Closesapache#16685Closesapache#16692Closesapache#16995Closesapache#17181Closesapache#17211Closesapache#17235Closesapache#17237Closesapache#17248Closesapache#17341Closesapache#17708Closesapache#17716Closesapache#17721Closesapache#17937
Added:
Closesapache#14739Closesapache#17139Closesapache#17445Closesapache#18042Closesapache#18359
Added:
Closesapache#16450Closesapache#16525Closesapache#17738
Added:
Closesapache#16458Closesapache#16508Closesapache#17714
Added:
Closesapache#17830Closesapache#14742
## How was this patch tested?
N/A
Author: hyukjinkwon <gurwls223@gmail.com>
Closesapache#18417 from HyukjinKwon/close-stale-pr.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@hvanhovell@SparkQA@HyukjinKwon@jiangxb1987@cloud-fan@gatorsmile