Skip to content

[SPARK-28599][SQL] Fix Execution Time and Duration column sorting for ThriftServerSessionPage - #25892

Closed
wangyum wants to merge 1 commit into
apache:masterfrom
wangyum:SPARK-28599
Closed

[SPARK-28599][SQL] Fix Execution Time and Duration column sorting for ThriftServerSessionPage#25892
wangyum wants to merge 1 commit into
apache:masterfrom
wangyum:SPARK-28599

Conversation

@wangyum

@wangyumwangyum commented Sep 22, 2019•
edited by dongjoon-hyun
Loading

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

This PR add support sorting Execution Time and Duration columns for ThriftServerSessionPage.

Why are the changes needed?

Previously, it's not sorted correctly.

Does this PR introduce any user-facing change?

Yes.

How was this patch tested?

Manually do the following and test sorting on those columns in the Spark Thrift Server Session Page.

$ sbin/start-thriftserver.sh
$ bin/beeline -u jdbc:hive2://localhost:10000
0: jdbc:hive2://localhost:10000> create table t(a int);
+---------+--+
| Result |
+---------+--+
+---------+--+
No rows selected (0.521 seconds)
0: jdbc:hive2://localhost:10000> select * from t;
+----+--+
| a |
+----+--+
+----+--+
No rows selected (0.772 seconds)
0: jdbc:hive2://localhost:10000> show databases;
+---------------+--+
| databaseName |
+---------------+--+
| default |
+---------------+--+
1 row selected (0.249 seconds)

Sorted by Execution Time column:
image

Sorted by Duration column:
image

@wangyum

wangyum commented Sep 22, 2019

Copy link
Copy Markdown
MemberAuthor

cc @amanomer

@amanomer

amanomer commented Sep 22, 2019

Copy link
Copy Markdown
Contributor

Looks good. Nice catch @wangyum .

@SparkQA

SparkQA commented Sep 22, 2019

Copy link
Copy Markdown

Test build #111154 has finished for PR 25892 at commit e794fe2.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@amanomeramanomer mentioned this pull request Sep 22, 2019
Closed
@amanomer

amanomer commented Sep 22, 2019•
edited
Loading

Copy link
Copy Markdown
Contributor

I would like to take this in branch-2.4 by #25882.

@dongjoon-hyun

dongjoon-hyun commented Sep 22, 2019•
edited
Loading

Copy link
Copy Markdown
Member

@amanomer and @wangyum .
It seems that idea is already reverted from #25882.

In general, please do not mix different ones. Especially, it's not a good idea when the original one is not merged yet. :)

@dongjoon-hyun

dongjoon-hyun commented Sep 22, 2019

Copy link
Copy Markdown
Member

BTW, @wangyum .
As a committer, please make it sure if the PR keeps the original four sections template of the PR.
For me, it seems that it's removed accidentally in this PR. It will be better if you recover it. 😄
cc @HyukjinKwon .

@amanomeramanomer mentioned this pull request Sep 22, 2019
Closed
@amanomer

amanomer commented Sep 22, 2019

Copy link
Copy Markdown
Contributor

Ok. I have created another PR #25893 to back-port this to branch-2.4.

@dongjoon-hyun

dongjoon-hyun commented Sep 22, 2019

Copy link
Copy Markdown
Member

Hi, @wangyum .
Did you check the original STS behavior?

@dongjoon-hyun

dongjoon-hyun commented Sep 22, 2019

Copy link
Copy Markdown
Member

In addition, this PR description is wrong. This is UI PR and that's the reason why you attach the screenshot. Could you fix the PR description?

Does this PR introduce any user-facing change?

No.

dongjoon-hyun
dongjoon-hyun approved these changes Sep 22, 2019
View reviewed changes

@dongjoon-hyundongjoon-hyun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that I fixed the PR description issues by

  1. Recovering the missing section
  2. Fixing UI section from NO to YES.
  3. Updating the test procedure for reviewers. (The section is for the reviewers mainly.)

+1, LGTM. Thank you, @wangyum and @amanomer .

@dongjoon-hyundongjoon-hyun changed the title [SPARK-28599][SQL] Support sorting Execution Time and Duration columns for ThriftServerSessionPage[SPARK-28599][SQL] Fix Execution Time and Duration column sorting for ThriftServerSessionPageSep 22, 2019
@dongjoon-hyun

dongjoon-hyun commented Sep 22, 2019

Copy link
Copy Markdown
Member

Merged to master.

@dongjoon-hyun

dongjoon-hyun commented Sep 22, 2019

Copy link
Copy Markdown
Member

I changed SPARK-28599 from Improvement to Bug.
Since this is a bug fix, we can backport this merged commit (with the fixed commit messages). However, in branch-2.4, there is no Execution Time column. There is Duration column only.
Please make it sure during backporting.

@wangyum
wangyum deleted the SPARK-28599 branch September 22, 2019 23:18
amanomer pushed a commit to amanomer/spark that referenced this pull request Sep 23, 2019
@wangyum@amanomer
[SPARK-28599][SQL] Fix Execution Time and Duration column sorting…
da43acc
… for ThriftServerSessionPage
This PR add support sorting `Execution Time` and `Duration` columns for `ThriftServerSessionPage`.
Previously, it's not sorted correctly.
Yes.
Manually do the following and test sorting on those columns in the Spark Thrift Server Session Page.
```
$ sbin/start-thriftserver.sh
$ bin/beeline -u jdbc:hive2://localhost:10000
0: jdbc:hive2://localhost:10000> create table t(a int);
+---------+--+
| Result |
+---------+--+
+---------+--+
No rows selected (0.521 seconds)
0: jdbc:hive2://localhost:10000> select * from t;
+----+--+
| a |
+----+--+
+----+--+
No rows selected (0.772 seconds)
0: jdbc:hive2://localhost:10000> show databases;
+---------------+--+
| databaseName |
+---------------+--+
| default |
+---------------+--+
1 row selected (0.249 seconds)
```
**Sorted by `Execution Time` column**:
![image](https://user-images.githubusercontent.com/5399861/65387476-53038900-dd7a-11e9-885c-fca80287f550.png)
**Sorted by `Duration` column**:
![image](https://user-images.githubusercontent.com/5399861/65387481-6e6e9400-dd7a-11e9-9318-f917247efaa8.png)
Closesapache#25892 from wangyum/SPARK-28599.
Authored-by: Yuming Wang <yumwang@ebay.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
@amanomeramanomer mentioned this pull request Sep 23, 2019
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dongjoon-hyundongjoon-hyundongjoon-hyun approved these changes

Assignees

No one assigned

Labels

SQL

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@wangyum@amanomer@SparkQA@dongjoon-hyun