Skip to content

TEZ-4250: Optimise TaskImpl::getCounters. - #295

Merged
abstractdog merged 4 commits into
apache:masterfrom
ayushtkn:TEZ-4250
Jun 23, 2023
Merged

TEZ-4250: Optimise TaskImpl::getCounters.#295
abstractdog merged 4 commits into
apache:masterfrom
ayushtkn:TEZ-4250

Conversation

@ayushtkn

Copy link
Copy Markdown
Member

No description provided.

Change-Id: I7db4db3a4d2cb70f2e1d96fbaa2f853524fbce8a
@tez-yetus

This comment was marked as outdated.

@tez-yetus

This comment was marked as outdated.

@tez-yetus

This comment was marked as outdated.

Change-Id: Ibaa0d6891658e1f54da883f9cc6b91eaa601c0ed
Change-Id: I1e36574b5fde496f52c155ff420d7434e067274f
@tez-yetus

This comment was marked as outdated.

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 19sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 1sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+1 💚mvninstall12m 2smaster passed
+1 💚compile0m 50smaster passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1
+1 💚compile0m 42smaster passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
+1 💚checkstyle0m 43smaster passed
+1 💚javadoc0m 26smaster passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1
+1 💚javadoc0m 23smaster passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
+0 🆗spotbugs1m 35sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs1m 33smaster passed
_ Patch Compile Tests _
+1 💚mvninstall0m 29sthe patch passed
+1 💚compile0m 34sthe patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1
+1 💚javac0m 34sthe patch passed
+1 💚compile0m 28sthe patch passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
+1 💚javac0m 28sthe patch passed
+1 💚checkstyle0m 24stez-dag: The patch generated 0 new + 127 unchanged - 1 fixed = 127 total (was 128)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 11sthe patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1
+1 💚javadoc0m 10sthe patch passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
+1 💚findbugs1m 18sthe patch passed
_ Other Tests _
+1 💚unit5m 16stez-dag in the patch passed.
+1 💚asflicense0m 11sThe patch does not generate ASF License warnings.
27m 14s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/5/artifact/out/Dockerfile
GITHUB PR#295
JIRA IssueTEZ-4250
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
unameLinux 8a4f0e695c1d 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / d20f334
Default JavaPrivate Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/5/testReport/
Max. process+thread count219 (vs. ulimit of 5500)
modulesC: tez-dag U: tez-dag
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/5/console
versionsgit=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@rbalamohan

Copy link
Copy Markdown
Contributor

LGTM. +1

TaskAttempt bestAttempt = selectBestAttempt();
if (bestAttempt != null) {
counters.incrAllCounters(bestAttempt.getCounters());
if (bestAttempt != null && tezCounters != null) {

@abstractdogabstractdogJun 23, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tezCounters != null only if speculation is enabled due to the recent patch, would it make sense to use that condition here for better readability? I guess it would clarify that we only use "this.counter" and this whole stuff in case of speculation
also, we're returning a non-null counter anyway, would this be clearer as:

TezCounters taskCounters = (bestAttempt != null) ? bestAttempt.getCounters() : TaskAttemptImpl.EMPTY_COUNTERS;
if (isSpeculationEnabled){
tezCounters.incrAllCounters(taskCounters); return tezCounters;
}
return taskCounters;

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@abstractdog

Copy link
Copy Markdown
Contributor

thanks for the review @rbalamohan
LGTM also, only a minor nit, let me know if it makes sense

Change-Id: I5f9fdc7b3e257b33d67cf1dd1fd2c4f0923a9541
@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 18sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+1 💚mvninstall12m 50smaster passed
+1 💚compile0m 46smaster passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1
+1 💚compile0m 43smaster passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
+1 💚checkstyle0m 48smaster passed
+1 💚javadoc0m 36smaster passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1
+1 💚javadoc0m 27smaster passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
+0 🆗spotbugs1m 47sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs1m 45smaster passed
_ Patch Compile Tests _
+1 💚mvninstall0m 27sthe patch passed
+1 💚compile0m 30sthe patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1
+1 💚javac0m 30sthe patch passed
+1 💚compile0m 27sthe patch passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
+1 💚javac0m 27sthe patch passed
+1 💚checkstyle0m 22stez-dag: The patch generated 0 new + 127 unchanged - 1 fixed = 127 total (was 128)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 10sthe patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1
+1 💚javadoc0m 10sthe patch passed with JDK Private Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
+1 💚findbugs1m 10sthe patch passed
_ Other Tests _
+1 💚unit5m 14stez-dag in the patch passed.
+1 💚asflicense0m 16sThe patch does not generate ASF License warnings.
28m 15s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/6/artifact/out/Dockerfile
GITHUB PR#295
JIRA IssueTEZ-4250
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
unameLinux aee43b4351c2 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / d20f334
Default JavaPrivate Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu122.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-gaus1-0ubuntu122.04-b09
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/6/testReport/
Max. process+thread count229 (vs. ulimit of 5500)
modulesC: tez-dag U: tez-dag
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-295/6/console
versionsgit=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@abstractdog
abstractdog merged commit 9aeb17b into apache:masterJun 23, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@ayushtkn@tez-yetus@rbalamohan@abstractdog