[SPARK-18665][SQL] set statement state to "ERROR" after user cancel job by cenyuhai · Pull Request #16099 · apache/spark

cenyuhai · 2016-12-01T12:21:41Z

What changes were proposed in this pull request?

set statement state to error after user canceled job

SparkQA · 2016-12-01T12:55:28Z

Test build #69472 has finished for PR 16099 at commit e3a9b08.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

rxin · 2016-12-01T18:39:56Z

cc @liancheng

SparkQA · 2016-12-02T06:29:12Z

Test build #69547 has finished for PR 16099 at commit 196ab66.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2016-12-06T09:00:41Z

Test build #69719 has finished for PR 16099 at commit ddb4128.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2016-12-06T09:49:48Z

Test build #69720 has finished for PR 16099 at commit 501d121.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2016-12-06T11:54:50Z

Test build #69727 has finished for PR 16099 at commit 40894a1.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

BruceXu1991 · 2017-05-16T09:24:38Z

 } catch {
 case e: HiveSQLException =>
+ HiveThriftServer2.listener.onStatementError(
+ statementId, e.getMessage, SparkUtils.exceptionString(e))


I tried your code. there seems have a bug.
e.getMessage may get a null value, which may cause NPE in ThriftServerPage.errorMessageCell(detail)

This PR still occurs in Spark 2.2.1. A workaround is to use "String.valueOf(e.getMessage)" instead of "e.getMessage" to avoid of NPE problem.

ok, but My pr is closed by community...

please catch the exception in ThriftServerPage.errorMessageCell. Without my pr, that function still throw exception.

gatorsmile · 2017-06-21T06:44:06Z

cc @cenyuhai

cenyuhai · 2018-02-05T04:20:02Z

@gatorsmile two years passed... I don't know what to say.

tooptoop4 · 2019-02-17T12:13:46Z

@cenyuhai want to open new pr?

## What changes were proposed in this pull request? This PR proposes to close stale PRs, mostly the same instances with apache#18017Closesapache#14085 - [SPARK-16408][SQL] SparkSQL Added file get Exception: is a directory … Closesapache#14239 - [SPARK-16593] [CORE] [WIP] Provide a pre-fetch mechanism to accelerate shuffle stage. Closesapache#14567 - [SPARK-16992][PYSPARK] Python Pep8 formatting and import reorganisation Closesapache#14579 - [SPARK-16921][PYSPARK] RDD/DataFrame persist()/cache() should return Python context managers Closesapache#14601 - [SPARK-13979][Core] Killed executor is re spawned without AWS key… Closesapache#14830 - [SPARK-16992][PYSPARK][DOCS] import sort and autopep8 on Pyspark examples Closesapache#14963 - [SPARK-16992][PYSPARK] Virtualenv for Pylint and pep8 in lint-python Closesapache#15227 - [SPARK-17655][SQL]Remove unused variables declarations and definations in a WholeStageCodeGened stage Closesapache#15240 - [SPARK-17556] [CORE] [SQL] Executor side broadcast for broadcast joins Closesapache#15405 - [SPARK-15917][CORE] Added support for number of executors in Standalone [WIP] Closesapache#16099 - [SPARK-18665][SQL] set statement state to "ERROR" after user cancel job Closesapache#16445 - [SPARK-19043][SQL]Make SparkSQLSessionManager more configurable Closesapache#16618 - [SPARK-14409][ML][WIP] Add RankingEvaluator Closesapache#16766 - [SPARK-19426][SQL] Custom coalesce for Dataset Closesapache#16832 - [SPARK-19490][SQL] ignore case sensitivity when filtering hive partition columns Closesapache#17052 - [SPARK-19690][SS] Join a streaming DataFrame with a batch DataFrame which has an aggregation may not work Closesapache#17267 - [SPARK-19926][PYSPARK] Make pyspark exception more user-friendly Closesapache#17371 - [SPARK-19903][PYSPARK][SS] window operator miss the `watermark` metadata of time column Closesapache#17401 - [SPARK-18364][YARN] Expose metrics for YarnShuffleService Closesapache#17519 - [SPARK-15352][Doc] follow-up: add configuration docs for topology-aware block replication Closesapache#17530 - [SPARK-5158] Access kerberized HDFS from Spark standalone Closesapache#17854 - [SPARK-20564][Deploy] Reduce massive executor failures when executor count is large (>2000) Closesapache#17979 - [SPARK-19320][MESOS][WIP]allow specifying a hard limit on number of gpus required in each spark executor when running on mesos Closesapache#18127 - [SPARK-6628][SQL][Branch-2.1] Fix ClassCastException when executing sql statement 'insert into' on hbase table Closesapache#18236 - [SPARK-21015] Check field name is not null and empty in GenericRowWit… Closesapache#18269 - [SPARK-21056][SQL] Use at most one spark job to list files in InMemoryFileIndex Closesapache#18328 - [SPARK-21121][SQL] Support changing storage level via the spark.sql.inMemoryColumnarStorage.level variable Closesapache#18354 - [SPARK-18016][SQL][CATALYST][BRANCH-2.1] Code Generation: Constant Pool Limit - Class Splitting Closesapache#18383 - [SPARK-21167][SS] Set kafka clientId while fetch messages Closesapache#18414 - [SPARK-21169] [core] Make sure to update application status to RUNNING if executors are accepted and RUNNING after recovery Closesapache#18432 - resolve com.esotericsoftware.kryo.KryoException Closesapache#18490 - [SPARK-21269][Core][WIP] Fix FetchFailedException when enable maxReqSizeShuffleToMem and KryoSerializer Closesapache#18585 - SPARK-21359 Closesapache#18609 - Spark SQL merge small files to big files Update InsertIntoHiveTable.scala Added: Closesapache#18308 - [SPARK-21099][Spark Core] INFO Log Message Using Incorrect Executor I… Closesapache#18599 - [SPARK-21372] spark writes one log file even I set the number of spark_rotate_log to 0 Closesapache#18619 - [SPARK-21397][BUILD]Maven shade plugin adding dependency-reduced-pom.xml to … Closesapache#18667 - Fix the simpleString used in error messages Closesapache#18782 - Branch 2.1 Added: Closesapache#17694 - [SPARK-12717][PYSPARK] Resolving race condition with pyspark broadcasts when using multiple threads Added: Closesapache#16456 - [SPARK-18994] clean up the local directories for application in future by annother thread Closesapache#18683 - [SPARK-21474][CORE] Make number of parallel fetches from a reducer configurable Closesapache#18690 - [SPARK-21334][CORE] Add metrics reporting service to External Shuffle Server Added: Closesapache#18827 - Merge pull request 1 from apache/master ## How was this patch tested? N/A Author: hyukjinkwon <gurwls223@gmail.com> Closesapache#18780 from HyukjinKwon/close-prs.

set statement state to error after user canceled job
e3a9b08

cenyuhai changed the title ~~[SPARK-18665][SQL] set statement state to error after user canceled job~~[SPARK-18665][SQL] set statement state to "ERROR" after user canceled jobDec 1, 2016

cenyuhai changed the title ~~[SPARK-18665][SQL] set statement state to "ERROR" after user canceled job~~[SPARK-18665][SQL] set statement state to "ERROR" after user cancel jobDec 1, 2016

change statement state by the current state
196ab66

fix bug when transition status from "CLOSED" to "ERROR"
ddb4128

fix bug when transition status from "CLOSED" to "FINISHED"
501d121

fix error when job is canceled, but job is still running
40894a1

BruceXu1991 reviewed May 16, 2017
View reviewed changes

HyukjinKwon mentioned this pull request Jul 31, 2017
[INFRA] Close stale PRs #18780
Closed

asfgit closed this in 3a45c7fAug 5, 2017

Navigation Menu

Uh oh!

[SPARK-18665][SQL] set statement state to "ERROR" after user cancel job - #16099

[SPARK-18665][SQL] set statement state to "ERROR" after user cancel job#16099
cenyuhai wants to merge 5 commits into
apache:masterfrom
cenyuhai:spark-18665

cenyuhai commented Dec 1, 2016

Uh oh!

SparkQA commented Dec 1, 2016

Uh oh!

rxin commented Dec 1, 2016

Uh oh!

SparkQA commented Dec 2, 2016

Uh oh!

SparkQA commented Dec 6, 2016

Uh oh!

SparkQA commented Dec 6, 2016

Uh oh!

SparkQA commented Dec 6, 2016

Uh oh!

BruceXu1991May 16, 2017

Uh oh!

BruceXu1991Feb 5, 2018

Uh oh!

cenyuhaiFeb 5, 2018

Uh oh!

cenyuhaiFeb 5, 2018

Uh oh!

gatorsmile commented Jun 21, 2017

Uh oh!

cenyuhai commented Feb 5, 2018

Uh oh!

tooptoop4 commented Feb 17, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

cenyuhai commented Dec 1, 2016

What changes were proposed in this pull request?

Uh oh!

SparkQA commented Dec 1, 2016

Uh oh!

rxin commented Dec 1, 2016

Uh oh!

SparkQA commented Dec 2, 2016

Uh oh!

SparkQA commented Dec 6, 2016

Uh oh!

SparkQA commented Dec 6, 2016

Uh oh!

SparkQA commented Dec 6, 2016

Uh oh!

BruceXu1991May 16, 2017

Choose a reason for hiding this comment

Uh oh!

BruceXu1991Feb 5, 2018

Choose a reason for hiding this comment

Uh oh!

cenyuhaiFeb 5, 2018

Choose a reason for hiding this comment

Uh oh!

cenyuhaiFeb 5, 2018

Choose a reason for hiding this comment

Uh oh!

gatorsmile commented Jun 21, 2017

Uh oh!

cenyuhai commented Feb 5, 2018

Uh oh!

tooptoop4 commented Feb 17, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants