Skip to content

TEZ-4227 Introduce convenient methods in TezID subclasses - #166

Merged
abstractdog merged 1 commit into
apache:masterfrom
ghanko:master
Feb 10, 2022
Merged

TEZ-4227 Introduce convenient methods in TezID subclasses#166
abstractdog merged 1 commit into
apache:masterfrom
ghanko:master

Conversation

@ghanko

Copy link
Copy Markdown
Contributor

Change-Id: I6cabfa75e9b6b62e41ba8c2cc5e3d2d1a8a49102

@tez-yetus

This comment has been minimized.

@tez-yetus

This comment was marked as outdated.

for (Vertex destVertex : outputVertexEdgeMap.keySet()) {
if (vertexAlreadyScheduled(destVertex)) { // Nothing to do if already scheduled.
} else {
if (!vertexAlreadyScheduled(destVertex)) { // Nothing to do if already scheduled.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can delete the comment "Nothing to do" from here as it corresponds to the deleted branch

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

import org.apache.hadoop.yarn.api.records.ApplicationId;

public interface DAGIDAware {
TezDAGID getDAGId();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

even if it looks strange (too many capitals), this might be changed to

getDAGID();

to stay consistent with others, so it should suggest that a TezID Object is returned here which is "ID", not an integer, which is id

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@tez-yetus

This comment was marked as outdated.

@tez-yetus

This comment was marked as outdated.

Change-Id: I6cabfa75e9b6b62e41ba8c2cc5e3d2d1a8a49102
@tez-yetus

This comment was marked as outdated.

@tez-yetus

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec13m 2sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 1sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 20 new or modified test files.
_ master Compile Tests _
+0 🆗mvndep5m 1sMaven dependency ordering for branch
+1 💚mvninstall9m 12smaster passed
+1 💚compile6m 19smaster passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚compile6m 4smaster passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚checkstyle6m 1smaster passed
+1 💚javadoc5m 48smaster passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚javadoc5m 18smaster passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+0 🆗spotbugs0m 45sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs9m 12smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 9sMaven dependency ordering for patch
+1 💚mvninstall4m 12sthe patch passed
+1 💚compile3m 43sthe patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚javac3m 43sthe patch passed
+1 💚compile3m 25sthe patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚javac3m 25sthe patch passed
-0 ⚠️checkstyle0m 10stez-common: The patch generated 16 new + 15 unchanged - 0 fixed = 31 total (was 15)
+1 💚checkstyle0m 12stez-runtime-internals: The patch generated 0 new + 33 unchanged - 1 fixed = 33 total (was 34)
+1 💚checkstyle0m 13stez-mapreduce: The patch generated 0 new + 55 unchanged - 1 fixed = 55 total (was 56)
-0 ⚠️checkstyle1m 7stez-dag: The patch generated 20 new + 1783 unchanged - 20 fixed = 1803 total (was 1803)
+1 💚checkstyle0m 13stez-tests: The patch generated 0 new + 34 unchanged - 2 fixed = 34 total (was 36)
+1 💚checkstyle0m 10sThe patch passed checkstyle in tez-ext-service-tests
+1 💚checkstyle0m 10sThe patch passed checkstyle in tez-protobuf-history-plugin
-0 ⚠️checkstyle0m 11stez-plugins/tez-yarn-timeline-history: The patch generated 1 new + 58 unchanged - 2 fixed = 59 total (was 60)
+1 💚checkstyle0m 10sThe patch passed checkstyle in tez-yarn-timeline-history-with-acls
+1 💚checkstyle0m 9sThe patch passed checkstyle in tez-yarn-timeline-cache-plugin
-0 ⚠️checkstyle0m 10stez-plugins/tez-yarn-timeline-history-with-fs: The patch generated 1 new + 35 unchanged - 1 fixed = 36 total (was 36)
+1 💚checkstyle0m 10sThe patch passed checkstyle in tez-history-parser
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc2m 59sthe patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚javadoc2m 42sthe patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚findbugs7m 55sthe patch passed
_ Other Tests _
+1 💚unit0m 34stez-common in the patch passed.
+1 💚unit0m 34stez-runtime-internals in the patch passed.
+1 💚unit1m 16stez-mapreduce in the patch passed.
+1 💚unit4m 25stez-dag in the patch passed.
+1 💚unit38m 50stez-tests in the patch passed.
+1 💚unit4m 4stez-ext-service-tests in the patch passed.
+1 💚unit0m 27stez-protobuf-history-plugin in the patch passed.
+1 💚unit1m 36stez-yarn-timeline-history in the patch passed.
+1 💚unit1m 50stez-yarn-timeline-history-with-acls in the patch passed.
+1 💚unit0m 18stez-yarn-timeline-cache-plugin in the patch passed.
+1 💚unit1m 35stez-yarn-timeline-history-with-fs in the patch passed.
+1 💚unit2m 29stez-history-parser in the patch passed.
+1 💚asflicense2m 22sThe patch does not generate ASF License warnings.
159m 2s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-166/6/artifact/out/Dockerfile
GITHUB PR#166
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
unameLinux 11e5938b2063 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / 6d7ef20
Default JavaPrivate Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
checkstylehttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-166/6/artifact/out/diff-checkstyle-tez-common.txt
checkstylehttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-166/6/artifact/out/diff-checkstyle-tez-dag.txt
checkstylehttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-166/6/artifact/out/diff-checkstyle-tez-plugins_tez-yarn-timeline-history.txt
checkstylehttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-166/6/artifact/out/diff-checkstyle-tez-plugins_tez-yarn-timeline-history-with-fs.txt
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-166/6/testReport/
Max. process+thread count1412 (vs. ulimit of 5500)
modulesC: tez-common tez-runtime-internals tez-mapreduce tez-dag tez-tests tez-ext-service-tests tez-plugins/tez-protobuf-history-plugin tez-plugins/tez-yarn-timeline-history tez-plugins/tez-yarn-timeline-history-with-acls tez-plugins/tez-yarn-timeline-cache-plugin tez-plugins/tez-yarn-timeline-history-with-fs tez-plugins/tez-history-parser U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-166/6/console
versionsgit=2.25.1 maven=3.6.3 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@abstractdogabstractdog left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 after addressing comments, this patch looks good to me, approving
I like the introduction of *IDAware interfaces, also having a consistent letter case: "ID" for TezID objects, "Id" for the integer inside
as the patch touches lots of code, let me give another 24-48h for others to tell their opinion (cc: @rbalamohan, @jteagles)

@ghanko: could you please check how the patch applies to branch-0.9? I'm pretty sure it has a bunch of conflicts, but I'm hoping all of them are obvious...if we could have the patch on branch-0.9, it helps to reduce the divergence between master/branch-0.9...only if it doesn't take too much time

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@ghanko@tez-yetus@abstractdog