Skip to content

increase more logs, and return appropriate error to front-end - #1051

Closed
prabhjyotsingh wants to merge 2 commits into
apache:masterfrom
prabhjyotsingh:ZEPPELIN-1036
Closed

increase more logs, and return appropriate error to front-end#1051
prabhjyotsingh wants to merge 2 commits into
apache:masterfrom
prabhjyotsingh:ZEPPELIN-1036

Conversation

@prabhjyotsingh

Copy link
Copy Markdown
Contributor

What is this PR for?

In the current implementation JDBC interpret only catches SQLException and ClassNotFoundException. However, there can be a case for example NullPointerException or for that matter anything else can be thrown.

What type of PR is it?

[Improvement]

Todos

  • - increase more logs, and return appropriate error to front-end

What is the Jira issue?

How should this be tested?

As an example for hive2.

Properties

namevalue
default.driverorg.apache.hive.jdbc.HiveDriver
default.urljdbc:hive2://hive-machine:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2

Dependencies

artifactexclude
org.apache.hive:hive-jdbc:1.2.1
org.apache.hadoop:hadoop-common:2.7.2

With the above setting used on an older driver for hive2 it shows exception in log, but on the front-end paragraph shows error, and no logs.

ERROR [2016-06-19 14:47:46,186] ({pool-2-thread-2} Job.java[run]:182) - Job failed
java.lang.NullPointerException
at org.apache.thrift.transport.TSocket.open(TSocket.java:168)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:248)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:190)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:187)
at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnection(JDBCInterpreter.java:218)
at org.apache.zeppelin.jdbc.JDBCInterpreter.getStatement(JDBCInterpreter.java:229)
at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:288)
at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:393)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341)
at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

Screenshots (if appropriate)

Before:
screen shot 2016-06-20 at 5 11 59 pm

After:
screen shot 2016-06-20 at 5 10 26 pm

Questions:

  • Does the licenses files need update? no
  • Is there breaking changes for older versions? no
  • Does this needs documentation? no

logger.error("Cannot run " + sql, e);
if (e.getMessage() == null) {
return new InterpreterResult(Code.ERROR, StringUtils.join(e.getStackTrace(), "\n"));
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about adding e.getStackTrace() if e.getMessage() != null?

@prabhjyotsingh

Copy link
Copy Markdown
ContributorAuthor

@jongyoul Thank you for the review, have made relevant change.

@felixcheung

Copy link
Copy Markdown
Member

LGTM

@prabhjyotsingh

Copy link
Copy Markdown
ContributorAuthor

CI fails for

#5104.1

Results :
Failed tests: ZeppelinSparkClusterTest.sparkRTest:104 expected:<FINISHED> but was:<ERROR>

#5104.7

Failed tests: ParagraphActionsIT.testTitleButton:343 After Show Title : The title field contains
Expected: "Untitled"
but: was ""
Tests in error: ParagraphActionsIT.testRemoveButton:143->AbstractZeppelinIT.waitForParagraph:88->AbstractZeppelinIT.pollingWait:114 » Timeout
ParagraphActionsIT.testMoveUpAndDown:180->AbstractZeppelinIT.waitForParagraph:88->AbstractZeppelinIT.pollingWait:114 » Timeout
ParagraphActionsIT.testCreateNewButton:75->AbstractZeppelinIT.waitForParagraph:88->AbstractZeppelinIT.pollingWait:114 » Timeout
Tests run: 15, Failures: 1, Errors: 3, Skipped: 0

Which looks unrelated.

@prabhjyotsingh

Copy link
Copy Markdown
ContributorAuthor

Merging this if no more discussion.

asfgit pushed a commit that referenced this pull request Jun 22, 2016
### What is this PR for?
In the current implementation JDBC interpret only catches SQLException and ClassNotFoundException. However, there can be a case for example NullPointerException or for that matter anything else can be thrown.
### What type of PR is it?
[Improvement]
### Todos
* [x] - increase more logs, and return appropriate error to front-end
### What is the Jira issue?
* Open an issue on Jira https://issues.apache.org/jira/browse/ZEPPELIN/
* Put link here, and add [ZEPPELIN-*Jira number*] in PR title, eg. [ZEPPELIN-533]
### How should this be tested?
As an example for hive2.
#### Properties
|name|	value|
| ------------- |:-------------:|
|default.driver|	org.apache.hive.jdbc.HiveDriver|
|default.url	|jdbc:hive2://hive-machine:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2|
#### Dependencies
|artifact	|exclude|
| ------------- |:-------------:|
|org.apache.hive:hive-jdbc:1.2.1	||
|org.apache.hadoop:hadoop-common:2.7.2||
With the above setting used on an older driver for hive2 it shows exception in log, but on the front-end paragraph shows error, and no logs.
```
ERROR [2016-06-19 14:47:46,186] ({pool-2-thread-2} Job.java[run]:182) - Job failed
java.lang.NullPointerException
at org.apache.thrift.transport.TSocket.open(TSocket.java:168)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:248)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:190)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:187)
at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnection(JDBCInterpreter.java:218)
at org.apache.zeppelin.jdbc.JDBCInterpreter.getStatement(JDBCInterpreter.java:229)
at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:288)
at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:393)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341)
at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
```
### Screenshots (if appropriate)
Before:
<img width="1436" alt="screen shot 2016-06-20 at 5 11 59 pm" src="https://cloud.githubusercontent.com/assets/674497/16192988/20ce0312-370a-11e6-967e-281e99dbb18f.png">
After:
<img width="1435" alt="screen shot 2016-06-20 at 5 10 26 pm" src="https://cloud.githubusercontent.com/assets/674497/16192984/1121d222-370a-11e6-8da9-43adc80c9c15.png">
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Prabhjyot Singh <prabhjyotsingh@gmail.com>
Closes#1051 from prabhjyotsingh/ZEPPELIN-1036 and squashes the following commits:
595ebc6 [Prabhjyot Singh] have e.getStackTrace() in all cases.
2e8dc0c [Prabhjyot Singh] increase more logs, and return appropriate error to front-end
(cherry picked from commit ab5ce62)
Signed-off-by: Prabhjyot Singh <prabhjyotsingh@gmail.com>
@prabhjyotsingh
prabhjyotsingh deleted the ZEPPELIN-1036 branch August 15, 2017 19:37
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@prabhjyotsingh@felixcheung@jongyoul