[SPARK-31967][UI] Downgrade to vis.js 4.21.0 to fix Jobs UI loading time regression

[gengliangwang]

What changes were proposed in this pull request?

After #28192, the job list page becomes very slow.
For example, after the following operation, the UI loading can take >40 sec.

(1 to 1000).foreach(_ => sc.parallelize(1 to 10).collect) 

This is caused by a performance issue of vis-timeline. The serious issue affects both branch-3.0 and branch-2.4

I tried a different version 4.21.0 from https://cdnjs.com/libraries/vis
The infinite drawing issue seems also fixed if the zoom is disabled as default.

Why are the changes needed?

Fix the serious perf issue in web UI by falling back vis-timeline-graph2d to an ealier version.

Does this PR introduce any user-facing change?

Yes, fix the UI perf regression

How was this patch tested?

Manual test

[gengliangwang]

I didn't find vis-timeline-graph2d.min.css.map and vis-timeline-graph2d.min.js.map in https://cdnjs.com/libraries/vis, so I deleted them. It should be minor.

[sarutak]

I can't download the downgraded version of vis from https://cdnjs.com/libraries/vis.
Is it temporarily not available?

[gengliangwang]

@sarutak it works fine in my side:

[sarutak]

I can't still get from the URL but can get from https://almende.github.io/vis/ anyway.

[gengliangwang]

OK, the 4.21.0 package in https://almende.github.io/vis/ should be the same

[sarutak]

@gengliangwang I checked out the change and build but timeline-view wouldn't work.

I also tried replacing the library with the one from https://almende.github.io/vis/ and works.

[SparkQA]

Test build #123889 has finished for PR 28806 at commit 53ac823.

• This patch fails due to an unknown error code, -9.
• This patch merges cleanly.
• This patch adds no public classes.

[gengliangwang]

@sarutak ok I have replaced with the one from https://almende.github.io/vis/
Thanks for trying!

[sarutak]

The infinite drawing issue seems also fixed if the zoom is disabled as default.

Actually that is not fixed in 4.21.0.
I replaced vis included in this PR with the one from https://almende.github.io/vis/ and I reproduced the infinite rendering.

Maybe, we compromise either infinite redrawing issue or this performance issue...

[gengliangwang]

well, in my setup, if the "Enable zooming" checkbox is not clicked, the infinite redrawing won't happen.

Maybe, we compromise either infinite redrawing issue or this performance issue...

yeah, I prefer infinite redrawing to the 40 seconds perf issue ...

[sarutak]

LGTM.
The updated version seems to work fine.
Pending for Jenkins.

[SparkQA]

Test build #123900 has finished for PR 28806 at commit f90f2b4.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[sarutak]

@gengliangwang Do you want to wait for other reviews?

[srowen]

Just for the record - this is a downgrade, upgrade?

[gengliangwang]

Just for the record - this is a downgrade, upgrade?

@srowen it's downgrade

[gengliangwang]

I am going to merge this one, and backport it to 3.0 / 2.4
@srowen@dongjoon-hyun@dbtsai@holdenk let me know if you have other concerns.

[dongjoon-hyun]

Hi, @gengliangwang and @sarutak . I'm wondering if this is safer enough than reverting SPARK-31420. Are we sure that there is no more UI regressions? It's difficult to detect manually this kind of slowness regression.

cc @rxin and @holdenk since they are the recent release managers.
Also, cc @gatorsmile , @cloud-fan , @dbtsai

[gengliangwang]

@dongjoon-hyun the PR #28192 is to fix the infinite timeline redraw bug in https://issues.apache.org/jira/browse/SPARK-31420 . The bug will happen again if we simply reverting #28192 .

This downgraded solution is better. In my local setup, the perf issue can't not be reproduced, and by default the infinite timeline redrawing won't happen unless the "Enable zooming" checkbox. I believe this is a better solution.

[gengliangwang]

Talked to @dongjoon-hyun offline. He is ok to merge this now

[gengliangwang]

Merging to master.

[sarutak]

This downgraded solution is better. In my local setup, the perf issue can't not be reproduced, and by default the infinite timeline redrawing won't happen unless the "Enable zooming" checkbox. I believe this is a better solution.

Downgrading vis.js is better though, actually, infinite redrawing can happen even if Enable zooming is not checked.
#28806 (comment) .
It seems to depend on browser.