Skip to content

TEZ-4411: Update FileSaver dependency - #206

Merged
abstractdog merged 2 commits into
apache:masterfrom
guptanikhil007:guptan/tez_4411
Apr 29, 2022
Merged

TEZ-4411: Update FileSaver dependency#206
abstractdog merged 2 commits into
apache:masterfrom
guptanikhil007:guptan/tez_4411

Conversation

@guptanikhil007

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Update FileSaver dependency to latest bower supported dependency

Why are the changes needed?

Previous dependency files were no longer available

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manual tests.

@guptanikhil007

Copy link
Copy Markdown
ContributorAuthor

Solution Credits:
@AnmolSun (Anmol Sundaram)

@abstractdog Please help with the review and merge as this is a blocker for other commits.

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec17m 50sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
-1 ❌mvninstall14m 1sroot in master failed.
+1 💚javadoc0m 55smaster passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚javadoc0m 28smaster passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
_ Patch Compile Tests _
+1 💚mvninstall2m 37sthe patch passed
+1 💚jshint32m 49sThere were no new jshint issues.
+1 💚whitespace0m 1sThe patch has no whitespace issues.
+1 💚javadoc0m 27sthe patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚javadoc0m 27sthe patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
_ Other Tests _
+1 💚unit2m 10stez-ui in the patch passed.
+1 💚asflicense0m 36sThe patch does not generate ASF License warnings.
73m 6s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-206/1/artifact/out/Dockerfile
GITHUB PR#206
JIRA IssueTEZ-4411
Optional Testsdupname asflicense javac javadoc unit jshint
unameLinux 02ca00ae28d9 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / 9f8d6fb
Default JavaPrivate Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
mvninstallhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-206/1/artifact/out/branch-mvninstall-root.txt
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-206/1/testReport/
Max. process+thread count88 (vs. ulimit of 5500)
modulesC: tez-ui U: tez-ui
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-206/1/console
versionsgit=2.25.1 maven=3.6.3 jshint=2.12.0
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@abstractdog

Copy link
Copy Markdown
Contributor

thanks @guptanikhil007
I can see tez-ui is still failing with your patch, could you please check?
https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-206/1/artifact/out/branch-mvninstall-root.txt
I can still see occurrences of "Teleborder" in the log file

@guptanikhil007

guptanikhil007 commented Apr 27, 2022

Copy link
Copy Markdown
ContributorAuthor

@abstractdog
I checked the build locally and it seems everything is working as expected.

There may be an issue with pre-commit pipeline.

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for tez 0.10.2-SNAPSHOT:
[INFO]
[INFO] tez ................................................ SUCCESS [ 30.072 s]
[INFO] hadoop-shim ........................................ SUCCESS [ 10.241 s]
[INFO] tez-api ............................................ SUCCESS [ 52.557 s]
[INFO] tez-build-tools .................................... SUCCESS [ 0.162 s]
[INFO] tez-common ......................................... SUCCESS [ 3.010 s]
[INFO] tez-runtime-internals .............................. SUCCESS [ 7.538 s]
[INFO] tez-runtime-library ................................ SUCCESS [ 8.932 s]
[INFO] tez-mapreduce ...................................... SUCCESS [ 7.238 s]
[INFO] tez-examples ....................................... SUCCESS [ 3.170 s]
[INFO] tez-dag ............................................ SUCCESS [ 20.194 s]
[INFO] tez-tests .......................................... SUCCESS [ 14.497 s]
[INFO] tez-ext-service-tests .............................. SUCCESS [ 3.875 s]
[INFO] tez-ui ............................................. SUCCESS [02:53 min]
[INFO] tez-plugins ........................................ SUCCESS [ 0.058 s]
[INFO] tez-protobuf-history-plugin ........................ SUCCESS [ 4.565 s]
[INFO] tez-yarn-timeline-history .......................... SUCCESS [ 5.371 s]
[INFO] tez-yarn-timeline-history-with-acls ................ SUCCESS [ 3.894 s]
[INFO] tez-yarn-timeline-cache-plugin ..................... SUCCESS [ 26.645 s]
[INFO] tez-yarn-timeline-history-with-fs .................. SUCCESS [ 3.879 s]
[INFO] tez-history-parser ................................. SUCCESS [ 31.786 s]
[INFO] tez-aux-services ................................... SUCCESS [ 38.671 s]
[INFO] tez-tools .......................................... SUCCESS [ 0.060 s]
[INFO] tez-perf-analyzer .................................. SUCCESS [ 0.045 s]
[INFO] tez-job-analyzer ................................... SUCCESS [ 3.656 s]
[INFO] tez-javadoc-tools .................................. SUCCESS [ 1.197 s]
[INFO] hadoop-shim-impls .................................. SUCCESS [ 0.042 s]
[INFO] hadoop-shim-2.8 .................................... SUCCESS [ 1.382 s]
[INFO] tez-dist ........................................... SUCCESS [ 47.991 s]
[INFO] Tez ................................................ SUCCESS [ 17.688 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 08:43 min
[INFO] Finished at: 2022-04-27T23:15:54+05:30
[INFO] ------------------------------------------------------------------------

@guptanikhil007

Copy link
Copy Markdown
ContributorAuthor

There is a failure on master but patch is passing:

[2022-04-27T05:02:43.468Z] | Vote | Subsystem | Runtime | Comment
[2022-04-27T05:02:43.468Z] ============================================================================
[2022-04-27T05:02:43.468Z] | 0 | reexec | 17m 50s | Docker mode activated. [2022-04-27T05:02:43.468Z] +---------------------------------------------------------------------------
[2022-04-27T05:02:43.468Z] | | | | Prechecks [2022-04-27T05:02:43.468Z] +---------------------------------------------------------------------------
[2022-04-27T05:02:43.468Z] | +1 | dupname | 0m 0s | No case conflicting files found. [2022-04-27T05:02:43.468Z] | +1 | @author | 0m 0s | The patch does not contain any @author [2022-04-27T05:02:43.468Z] | | | | tags.
[2022-04-27T05:02:43.468Z] | -1 | test4tests | 0m 0s | The patch doesn't appear to include any [2022-04-27T05:02:43.468Z] | | | | new or modified tests. Please justify why
[2022-04-27T05:02:43.468Z] | | | | no new tests are needed for this patch.
[2022-04-27T05:02:43.468Z] | | | | Also please list what manual steps were
[2022-04-27T05:02:43.468Z] | | | | performed to verify this patch.
[2022-04-27T05:02:43.468Z] +---------------------------------------------------------------------------
[2022-04-27T05:02:43.468Z] | | | | master Compile Tests [2022-04-27T05:02:43.468Z] +---------------------------------------------------------------------------
[2022-04-27T05:02:43.468Z] | -1 | mvninstall | 14m 1s | root in master failed. [2022-04-27T05:02:43.468Z] | +1 | javadoc | 0m 55s | master passed with JDK Private [2022-04-27T05:02:43.468Z] | | | | Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
[2022-04-27T05:02:43.468Z] | +1 | javadoc | 0m 28s | master passed with JDK Private [2022-04-27T05:02:43.468Z] | | | | Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b
[2022-04-27T05:02:43.468Z] | | | | 07
[2022-04-27T05:02:43.468Z] +---------------------------------------------------------------------------
[2022-04-27T05:02:43.468Z] | | | | Patch Compile Tests [2022-04-27T05:02:43.468Z] +---------------------------------------------------------------------------
[2022-04-27T05:02:43.468Z] | +1 | mvninstall | 2m 37s | the patch passed [2022-04-27T05:02:43.468Z] | +1 | jshint | 32m 49s | There were no new jshint issues. [2022-04-27T05:02:43.468Z] | +1 | whitespace | 0m 1s | The patch has no whitespace issues. [2022-04-27T05:02:43.468Z] | +1 | javadoc | 0m 27s | the patch passed with JDK Private [2022-04-27T05:02:43.468Z] | | | | Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
[2022-04-27T05:02:43.468Z] | +1 | javadoc | 0m 27s | the patch passed with JDK Private [2022-04-27T05:02:43.468Z] | | | | Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b
[2022-04-27T05:02:43.468Z] | | | | 07
[2022-04-27T05:02:43.468Z] +---------------------------------------------------------------------------
[2022-04-27T05:02:43.468Z] | | | | Other Tests [2022-04-27T05:02:43.468Z] +---------------------------------------------------------------------------
[2022-04-27T05:02:43.468Z] | +1 | unit | 2m 10s | tez-ui in the patch passed. [2022-04-27T05:02:43.468Z] | +1 | asflicense | 0m 36s | The patch does not generate ASF License [2022-04-27T05:02:43.468Z] | | | | warnings.
[2022-04-27T05:02:43.468Z] | | | 73m 6s | [2022-04-27T05:02:43.468Z] 

@abstractdog

Copy link
Copy Markdown
Contributor

@guptanikhil007 : I see, that's kind of my concern, as we want to fix precommit here too (not only local builds), I want to make sure that once we merge this change, precommit will work
are you aware of any functionalities that made precommit stuck with the Teleborder version?

I mean, what you attached:

[2022-04-27T05:02:43.468Z] | -1 | mvninstall | 14m 1s | root in master failed. 

...exactly belongs to the log file I mentioned above:
https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-206/1/artifact/out/branch-mvninstall-root.txt

precommit is supposed to pick up the source code changes in the pull request, I'm just looking for the reason why it failed, and tried to fetch Teleborder version

@guptanikhil007

guptanikhil007 commented Apr 27, 2022

Copy link
Copy Markdown
ContributorAuthor
[2022-04-27T05:02:43.468Z] | | | | master Compile Tests
[2022-04-27T05:02:43.468Z] | -1 | mvninstall | 14m 1s | root in master failed. [2022-04-27T05:02:43.468Z] +---------------------------------------------------------------------------
[2022-04-27T05:02:43.468Z] | | | | Patch Compile Tests [2022-04-27T05:02:43.468Z] +---------------------------------------------------------------------------
[2022-04-27T05:02:43.468Z] | +1 | mvninstall | 2m 37s | the patch passed 

@abstractdog
mvninstall on master will fail as the patch is not applied till this step.

Build Steps:

[2022-04-27T03:49:38.848Z] Confirming git environment
[2022-04-27T03:49:39.888Z] Re-execing under Docker
[2022-04-27T03:49:39.888Z] Removing old images
[2022-04-27T03:49:41.450Z] Docker Container Maintenance
[2022-04-27T03:49:41.450Z] Docker Image Creation
[2022-04-27T04:07:31.723Z] Confirming git environment
[2022-04-27T04:07:31.723Z] Re-exec mode detected. Continuing.
[2022-04-27T04:07:31.723Z] Determining needed tests
[2022-04-27T04:07:31.724Z] Checking for duplicated filenames that differ only in case
[2022-04-27T04:07:31.724Z] Checking for @author tags: patch
[2022-04-27T04:07:31.724Z] Checking for long paths: patch
[2022-04-27T04:07:31.724Z] Checking there are new or changed tests in the patch.
[2022-04-27T04:07:31.725Z] maven install: master
[2022-04-27T04:21:40.159Z] jshint plugin: master
[2022-04-27T04:26:04.808Z] Pre-patch javadoc verification on master
[2022-04-27T04:27:29.565Z] Cleaning the source tree
[2022-04-27T04:27:38.497Z] Applying patch to master
[2022-04-27T04:27:39.020Z] maven install: patch
[2022-04-27T04:30:23.564Z] jshint plugin: patch
[2022-04-27T04:58:54.471Z] Checking for whitespace issues.
[2022-04-27T04:58:56.046Z] javadoc verification: patch
[2022-04-27T04:59:51.288Z] Running unit tests
[2022-04-27T05:02:09.607Z] Determining number of ASF License errors
[2022-04-27T05:02:38.226Z] Writing Brief Text Report to /precommit/brief.txt
[2022-04-27T05:02:38.751Z] Adding comment to Github
[2022-04-27T05:02:40.351Z] Adding comment to Gitlab
[2022-04-27T05:02:40.876Z] Writing HTML to /precommit/report.htm
[2022-04-27T05:02:41.401Z] Adding comment to JIRA

"0.1.1": "7a13f6dba5a340e1cb9e0b64c1c711e4d7edaca1"
},
"https://github.com/eligrey/FileSaver.js.git": {
"1.3.4": "b4a918669accb81f184c610d741a4a8e1306aa27"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would make sense to point to same commit (eligrey/FileSaver.js@b7cf622) in this PR so there are no extra changes.
Upgrading can be handled in separate commit.

@abstractdog

Copy link
Copy Markdown
Contributor

thanks @guptanikhil007, makes sense, so it should work after merging
please consider @Deependra-Patel 's comment regarding the version, otherwise, looks good to me

@guptanikhil007

Copy link
Copy Markdown
ContributorAuthor

The lowest bower version available is 1.2.0.

node_modules/bower/bin/bower install file-saver#v1.20150507.2 --save
bower file-saver#v1.20150507.2 not-cached https://github.com/eligrey/FileSaver.js.git#v1.20150507.2
bower file-saver#v1.20150507.2 resolve https://github.com/eligrey/FileSaver.js.git#v1.20150507.2
bower file-saver#v1.20150507.2 ENORESTARGET No version found that was able to satisfy v1.20150507.2
Additional error details:
Available versions: 1.2.0, 1.2.1, 1.2.2, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.6, 1.3.7, 1.3.8, 2.0.0, 2.0.4, 2.0.2, 2.0.4

@Deependra-Patel@abstractdog Should we go with 1.2.0?

@abstractdog

abstractdog commented Apr 29, 2022

Copy link
Copy Markdown
Contributor

The lowest bower version available is 1.2.0.

node_modules/bower/bin/bower install file-saver#v1.20150507.2 --save
bower file-saver#v1.20150507.2 not-cached https://github.com/eligrey/FileSaver.js.git#v1.20150507.2
bower file-saver#v1.20150507.2 resolve https://github.com/eligrey/FileSaver.js.git#v1.20150507.2
bower file-saver#v1.20150507.2 ENORESTARGET No version found that was able to satisfy v1.20150507.2
Additional error details:
Available versions: 1.2.0, 1.2.1, 1.2.2, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.6, 1.3.7, 1.3.8, 2.0.0, 2.0.4, 2.0.2, 2.0.4

@Deependra-Patel@abstractdog Should we go with 1.2.0?

I cannot tell honestly, I've just uploaded the one on my laptop to: https://issues.apache.org/jira/secure/attachment/13043100/FileSaver.js
so this is used to be downloaded most probably by bower from Teleborder, but there are couple of diffs with the current 1.2.0 too

instead of manually looking for changes, please do the following:

#download the latest 1.x from FileSaver.js
#go to master branch without this patch
#remove your current FileSave.js
cd tez-ui
rm ./src/main/webapp/bower_components/file-saver.js/FileSaver.js
mvn test #should fail as FileSaver.js is not present
#download a target FileSaver.js version and copy it destination
cp the_FileSaver_you_just_downloaded.js ./src/main/webapp/bower_components/file-saver.js/FileSaver.js
mvn test #should pass

if all tests pass, I'm fine with the version, no matter which 1.x

@guptanikhil007

Copy link
Copy Markdown
ContributorAuthor

mvn test is passing with 1.2.0

1..947
# tests 947
# pass 947
# fail 0
# ok

@abstractdog
abstractdog self-requested a review April 29, 2022 06:02
@abstractdog

Copy link
Copy Markdown
Contributor

thanks @guptanikhil007 , I'm about to merge this soon

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec2m 37sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
-1 ❌mvninstall15m 51sroot in master failed.
+1 💚javadoc1m 7smaster passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚javadoc0m 42smaster passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
_ Patch Compile Tests _
+1 💚mvninstall3m 13sthe patch passed
+1 💚jshint44m 58sThere were no new jshint issues.
+1 💚whitespace0m 1sThe patch has no whitespace issues.
+1 💚javadoc0m 33sthe patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚javadoc0m 33sthe patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
_ Other Tests _
+1 💚unit3m 21stez-ui in the patch passed.
+1 💚asflicense0m 47sThe patch does not generate ASF License warnings.
74m 44s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-206/2/artifact/out/Dockerfile
GITHUB PR#206
JIRA IssueTEZ-4411
Optional Testsdupname asflicense javac javadoc unit jshint
unameLinux d0cb5361bc9e 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24 17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / 9f8d6fb
Default JavaPrivate Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
mvninstallhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-206/2/artifact/out/branch-mvninstall-root.txt
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-206/2/testReport/
Max. process+thread count88 (vs. ulimit of 5500)
modulesC: tez-ui U: tez-ui
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-206/2/console
versionsgit=2.25.1 maven=3.6.3 jshint=2.12.0
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@abstractdog
abstractdog merged commit 2a9495a into apache:masterApr 29, 2022
@abstractdog

Copy link
Copy Markdown
Contributor

merged, created empty PR to check mvninstall phase: #207

@guptanikhil007
guptanikhil007 deleted the guptan/tez_4411 branch April 29, 2022 07:04
@Deependra-Patel

Copy link
Copy Markdown

The lowest bower version available is 1.2.0.

node_modules/bower/bin/bower install file-saver#v1.20150507.2 --save
bower file-saver#v1.20150507.2 not-cached https://github.com/eligrey/FileSaver.js.git#v1.20150507.2
bower file-saver#v1.20150507.2 resolve https://github.com/eligrey/FileSaver.js.git#v1.20150507.2
bower file-saver#v1.20150507.2 ENORESTARGET No version found that was able to satisfy v1.20150507.2
Additional error details:
Available versions: 1.2.0, 1.2.1, 1.2.2, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.6, 1.3.7, 1.3.8, 2.0.0, 2.0.4, 2.0.2, 2.0.4

@Deependra-Patel@abstractdog Should we go with 1.2.0?

What I meant was we could have kept the commit id b7cf622909258086bc63ad764d08fcaed780ab42 instead of b4a918669accb81f184c610d741a4a8e1306aa27 irrespective of what release version we use. That should probably checkout this repo to this specific commit, so everything remains same as before.

asfgit pushed a commit that referenced this pull request Apr 30, 2022
mudit1289 pushed a commit to mudit1289/tez that referenced this pull request Feb 25, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@guptanikhil007@tez-yetus@abstractdog@Deependra-Patel