[SPARK-28004][UI] Update jquery to 3.4.1 by srowen · Pull Request #24843 · apache/spark

srowen · 2019-06-11T19:39:21Z

What changes were proposed in this pull request?

We're using an old-ish jQuery, 1.12.4, and should probably update for Spark 3 to keep up in general, but also to keep up with CVEs. In fact, we know of at least one resolved in only 3.4.0+ (https://nvd.nist.gov/vuln/detail/CVE-2019-11358). They may not affect Spark, but, if the update isn't painful, maybe worthwhile in order to make future 3.x updates easier.

jQuery 1 -> 2 doesn't sound like a breaking change, as 2.0 is supposed to maintain compatibility with 1.9+ (https://blog.jquery.com/2013/04/18/jquery-2-0-released/)

2 -> 3 has breaking changes: https://jquery.com/upgrade-guide/3.0/. It's hard to evaluate each one, but the most likely area for problems is in ajax(). However, our usage of jQuery (and plugins) is pretty simple.

Update jquery to 3.4.1; update jquery blockUI and mustache to latest

How was this patch tested?

Manual testing of docs build (except R docs), worker/master UI, spark application UI.
Note: this really doesn't guarantee it works, as our tests can't test javascript, and this is merely anecdotal testing, although I clicked about every link I could find. There's a risk this breaks a minor part of the UI; it does seem to work fine in the main.

SparkQA · 2019-06-11T21:05:35Z

Test build #106394 has finished for PR 24843 at commit 185cac1.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

srowen · 2019-06-11T21:16:55Z

 return obj != null && typeof obj === 'object' && (propName in obj);
 }

+ /**


These changes are all from the source jquery.mustache.js, not custom modifications

I also checked this. This file is identical with mustache.js v3.0.1. Only the trailing spaces are removed in a few lines.

srowen · 2019-06-11T21:17:09Z

 // Scroll now too in case we had opened the page on a hash, but wait a bit because some browsers
 // will try to do *their* initial scroll after running the onReady handler.
- $(window).load(function() { setTimeout(function() { maybeScrollToHash(); }, 25); });
+ $(window).on('load', function() { setTimeout(function() { maybeScrollToHash(); }, 25); });


This is actually the only change I seemed to need to make to the Spark javascript.

Yep. This is the correct update based on the upgrade-guide.
https://jquery.com/upgrade-guide/3.0/#breaking-change-load-unload-and-error-removed

srowen · 2019-06-12T01:11:48Z

Ah right. The history server. There's a problem there. I'll look into it.

SparkQA · 2019-06-12T04:13:33Z

Test build #106400 has finished for PR 24843 at commit 8fe52eb.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-06-12T20:42:14Z

Test build #106441 has finished for PR 24843 at commit 48d3ca4.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-06-12T22:39:44Z

Test build #106443 has finished for PR 24843 at commit 3ea4db7.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-06-13T14:14:31Z

Test build #106465 has finished for PR 24843 at commit ebe85ba.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

srowen · 2019-06-13T14:18:39Z

Hm, for anyone following along, these tests pass locally. Maybe I'm doing something wrong running locally. Looks like an httpclient version mismatch that I'll have to run down: Cannot locate declared field class : org.apache.http.impl.client.HttpClientBuilder.dnsResolver

SparkQA · 2019-06-13T17:19:25Z

Test build #106471 has finished for PR 24843 at commit f1e02e4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2019-06-15T05:05:22Z


 var historySummary = $("#history-summary");
- var searchString = historySummary["context"]["location"]["search"];
+ var searchString = window.location.search;


I also verified that this works in the same way.

dongjoon-hyun

+1, LGTM. Thank you, @srowen !

This should be tested in the browser's private mode to avoid old javascript cache.
I verified this PR with the followings.

Spark shell application UI
Thrift JDBC/ODBC Server application UI
Spark History Server UI on my several Spark history logs.

All tabs look working correctly. I might miss a few tiny UI features like hover info, but the navigation and the info looks correct in general. I hope we can merge this and proceed.

Merged to master.

dongjoon-hyun · 2019-06-15T05:28:07Z

cc @gatorsmile and @gengliangwang

## What changes were proposed in this pull request? We're using an old-ish jQuery, 1.12.4, and should probably update for Spark 3 to keep up in general, but also to keep up with CVEs. In fact, we know of at least one resolved in only 3.4.0+ (https://nvd.nist.gov/vuln/detail/CVE-2019-11358). They may not affect Spark, but, if the update isn't painful, maybe worthwhile in order to make future 3.x updates easier. jQuery 1 -> 2 doesn't sound like a breaking change, as 2.0 is supposed to maintain compatibility with 1.9+ (https://blog.jquery.com/2013/04/18/jquery-2-0-released/) 2 -> 3 has breaking changes: https://jquery.com/upgrade-guide/3.0/. It's hard to evaluate each one, but the most likely area for problems is in ajax(). However, our usage of jQuery (and plugins) is pretty simple. Update jquery to 3.4.1; update jquery blockUI and mustache to latest ## How was this patch tested? Manual testing of docs build (except R docs), worker/master UI, spark application UI. Note: this really doesn't guarantee it works, as our tests can't test javascript, and this is merely anecdotal testing, although I clicked about every link I could find. There's a risk this breaks a minor part of the UI; it does seem to work fine in the main. Closesapache#24843 from srowen/SPARK-28004. Authored-by: Sean Owen <sean.owen@databricks.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>

srowen · 2019-06-15T14:44:38Z

- implicit val webDriver: WebDriver = new HtmlUnitDriver(true) {
- getWebClient.getOptions.setThrowExceptionOnScriptError(false)
- }
+ implicit val webDriver: WebDriver =


One last clarifying comment: This change slightly improves the test to not ignore javascript errors, and as part of that, needed to test as a modern browser (IE 11 vs IE 8).

gengliangwang · 2019-06-17T05:52:32Z

A late LGTM. I also tried grep -r "jquery-1.12.4.min.js" * and there is no matched results.

gabrieljones · 2019-10-07T19:05:48Z

any chance this will be back ported to 2.3? 2.3 is currently quarantined at my site because of this.

srowen · 2019-10-07T19:11:37Z

No, 2.3 is EOL. You should update to 2.4.x

We're using an old-ish jQuery, 1.12.4, and should probably update for Spark 3 to keep up in general, but also to keep up with CVEs. In fact, we know of at least one resolved in only 3.4.0+ (https://nvd.nist.gov/vuln/detail/CVE-2019-11358). They may not affect Spark, but, if the update isn't painful, maybe worthwhile in order to make future 3.x updates easier. jQuery 1 -> 2 doesn't sound like a breaking change, as 2.0 is supposed to maintain compatibility with 1.9+ (https://blog.jquery.com/2013/04/18/jquery-2-0-released/) 2 -> 3 has breaking changes: https://jquery.com/upgrade-guide/3.0/. It's hard to evaluate each one, but the most likely area for problems is in ajax(). However, our usage of jQuery (and plugins) is pretty simple. Update jquery to 3.4.1; update jquery blockUI and mustache to latest Manual testing of docs build (except R docs), worker/master UI, spark application UI. Note: this really doesn't guarantee it works, as our tests can't test javascript, and this is merely anecdotal testing, although I clicked about every link I could find. There's a risk this breaks a minor part of the UI; it does seem to work fine in the main. Closesapache#24843 from srowen/SPARK-28004. Authored-by: Sean Owen <sean.owen@databricks.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>

Update jquery to 3.4.1; update jquery blockUI and mustache to latest
185cac1

srowen self-assigned this Jun 11, 2019

srowen changed the title ~~[SPARK-28004][UI]~~ [SPARK-28004][UI] Update jquery to 3.4.1Jun 11, 2019

srowen commented Jun 11, 2019
View reviewed changes

Fix history server access to query string
8fe52eb

dongjoon-hyun added the IMPROVEMENT label Jun 12, 2019

Attempt to fix JS Selenium problem with htmlunit update
48d3ca4

Oops, build fix
3ea4db7

Try to get SBT to honor the htmlunit dependency update
ebe85ba

Also coerce httpclient version up for SBT
f1e02e4

dongjoon-hyun added WEB UI and removed IMPROVEMENT labels Jun 14, 2019

dongjoon-hyun reviewed Jun 15, 2019
View reviewed changes

dongjoon-hyun approved these changes Jun 15, 2019
View reviewed changes

dongjoon-hyun closed this in 15462e1Jun 15, 2019

srowen commented Jun 15, 2019
View reviewed changes

srowen deleted the SPARK-28004 branch June 19, 2019 13:55

srowen mentioned this pull request Sep 19, 2019
[MINOR][BUILD] Fix about 15 misc build warnings #25852
Closed

Uh oh!

Conversation

srowen commented Jun 11, 2019

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

SparkQA commented Jun 11, 2019

Uh oh!

srowenJun 11, 2019

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyunJun 15, 2019• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

srowenJun 11, 2019

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyunJun 15, 2019

Choose a reason for hiding this comment

Uh oh!

srowen commented Jun 12, 2019

Uh oh!

SparkQA commented Jun 12, 2019

Uh oh!

SparkQA commented Jun 12, 2019

Uh oh!

SparkQA commented Jun 12, 2019

Uh oh!

SparkQA commented Jun 13, 2019

Uh oh!

srowen commented Jun 13, 2019

Uh oh!

SparkQA commented Jun 13, 2019

Uh oh!

dongjoon-hyunJun 15, 2019

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun commented Jun 15, 2019

Uh oh!

srowenJun 15, 2019

Choose a reason for hiding this comment

Uh oh!

gengliangwang commented Jun 17, 2019

Uh oh!

gabrieljones commented Oct 7, 2019

Uh oh!

srowen commented Oct 7, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

dongjoon-hyunJun 15, 2019•
edited
Loading

dongjoon-hyun left a comment •
edited
Loading