Skip to content

[SPARK-40180][SQL] Format error messages by spark-sql - #37590

Closed
MaxGekk wants to merge 6 commits into
apache:masterfrom
MaxGekk:spark-sql-error-json
Closed

[SPARK-40180][SQL] Format error messages by spark-sql#37590
MaxGekk wants to merge 6 commits into
apache:masterfrom
MaxGekk:spark-sql-error-json

Conversation

@MaxGekk

@MaxGekkMaxGekk commented Aug 20, 2022

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

  1. Respect the SQL config spark.sql.error.messageFormat introduced by [SPARK-40098][SQL] Format error messages in the Thrift Server #37520, and output error messages in the one of format: PRETTY (by default), MINIMAL or STANDARD.
  2. In the PRETTY format, output the error message of the AnalysisException exception in the same way as for other exceptions, i. e. w/o Error in query:.
  3. Take into account the silent CLI option, and output the call stack only when it is false.

In the MINIMAL and STANDARD formats don't print the call stack independently from the silent mode.

Why are the changes needed?

To respect the SQL config spark.sql.error.messageFormat and to be consistent to error outputs of the Thrift Server.

Does this PR introduce any user-facing change?

Yes.

The PR changes the behavior for AnalysisException. In that case, spark-sql does not output the prefix: Error in query: by default (format is PRETTY).

Before:

spark-sql>DROPTABLEajhkdha;
Error in query: Table or view not found: ajhkdha; line1 pos 11;

After:

spark-sql>DROPTABLEajhkdha;
Table or view not found: ajhkdha; line1 pos 11;

How was this patch tested?

By running the modified test suites:

$ build/sbt -Phive-2.3 -Phive-thriftserver "test:testOnly org.apache.spark.sql.hive.thriftserver.CliSuite"

@MaxGekkMaxGekk changed the title [WIP][SQL] Format error messages by spark-sql[WIP][SPARK-40180][SQL] Format error messages by spark-sqlAug 22, 2022
@MaxGekkMaxGekk changed the title [WIP][SPARK-40180][SQL] Format error messages by spark-sql[SPARK-40180][SQL] Format error messages by spark-sqlAug 22, 2022
@MaxGekk
MaxGekk marked this pull request as ready for review August 22, 2022 20:34
@MaxGekk

Copy link
Copy Markdown
MemberAuthor

@srielau@anchovYu Please, take a look at the PR.

@srielausrielau left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice!

// to the console.
case _ => err.println(s"""Error in query: ${e.getMessage}""")
}
case _ => err.println(rc.getErrorMessage())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

even before this PR, we may not print Error in query: either. I think it's fine to clean it up and get rid of this prefix

@MaxGekk

Copy link
Copy Markdown
MemberAuthor

Merging to master. Thank you, @srielau@yaooqinn@cloud-fan for review.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@MaxGekk@cloud-fan@srielau@yaooqinn