Skip to content

TEZ-4605: Enable hadoop CallerContext in TezChild and DAGAppMaster - #394

Merged
ayushtkn merged 6 commits into
apache:masterfrom
liangyu-1:TEZ-4605
Feb 21, 2025
Merged

TEZ-4605: Enable hadoop CallerContext in TezChild and DAGAppMaster#394
ayushtkn merged 6 commits into
apache:masterfrom
liangyu-1:TEZ-4605

Conversation

@liangyu-1

@liangyu-1liangyu-1 commented Feb 20, 2025

Copy link
Copy Markdown
Contributor

As described in TEZ-4605

What did I change

I modified tez-dag/src/main/java/org/apache/tez/dag/app/DAGAppMaster.java and tez-runtime-internals/src/main/java/org/apache/tez/runtime/task/TezChild.java

Why did I change

I want to enable org.apache.hadoop.ipc.CallerContext when running TEZ on yarn, so that we can trace how the application interacted with hdfs files.

How did I test

I modified the code and build the whole project, then I test on my own hadoop cluster, we can see the callcontext information in hdfs-audit.log.

here is a screenshot of it:
image

@liangyu-1

liangyu-1 commented Feb 20, 2025

Copy link
Copy Markdown
ContributorAuthor

@tez-yetus@abstractdog
hi! would you please help me check this issue?

@tez-yetus

tez-yetus commented Feb 20, 2025

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec22m 40sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+0 🆗mvndep2m 4sMaven dependency ordering for branch
+1 💚mvninstall10m 9smaster passed
+1 💚compile1m 14smaster passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚compile1m 7smaster passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚checkstyle1m 11smaster passed
+1 💚javadoc0m 49smaster passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javadoc0m 41smaster passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+0 🆗spotbugs1m 23sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs2m 27smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for patch
+1 💚mvninstall0m 49sthe patch passed
+1 💚compile0m 53sthe patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javac0m 53sthe patch passed
+1 💚compile0m 47sthe patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚javac0m 47sthe patch passed
+1 💚checkstyle0m 30sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 19sthe patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javadoc0m 19sthe patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚findbugs2m 6sthe patch passed
_ Other Tests _
+1 💚unit0m 56stez-runtime-internals in the patch passed.
+1 💚unit5m 5stez-dag in the patch passed.
+1 💚asflicense0m 21sThe patch does not generate ASF License warnings.
56m 0s
SubsystemReport/Notes
DockerClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/1/artifact/out/Dockerfile
GITHUB PR#394
JIRA IssueTEZ-4605
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
unameLinux 177372eadf6f 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / 871866e
Default JavaPrivate Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/1/testReport/
Max. process+thread count305 (vs. ulimit of 5500)
modulesC: tez-runtime-internals tez-dag U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/1/console
versionsgit=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@tez-yetus

tez-yetus commented Feb 20, 2025

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec7m 31sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+0 🆗mvndep2m 4sMaven dependency ordering for branch
+1 💚mvninstall10m 9smaster passed
+1 💚compile1m 13smaster passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚compile1m 8smaster passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚checkstyle1m 9smaster passed
+1 💚javadoc0m 49smaster passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javadoc0m 41smaster passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+0 🆗spotbugs1m 24sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs2m 29smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 10sMaven dependency ordering for patch
+1 💚mvninstall0m 49sthe patch passed
+1 💚compile0m 51sthe patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javac0m 51sthe patch passed
+1 💚compile0m 45sthe patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚javac0m 45sthe patch passed
+1 💚checkstyle0m 31sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 19sthe patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javadoc0m 19sthe patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚findbugs2m 7sthe patch passed
_ Other Tests _
+1 💚unit0m 55stez-runtime-internals in the patch passed.
+1 💚unit5m 36stez-dag in the patch passed.
+1 💚asflicense0m 21sThe patch does not generate ASF License warnings.
41m 21s
SubsystemReport/Notes
DockerClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/2/artifact/out/Dockerfile
GITHUB PR#394
JIRA IssueTEZ-4605
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
unameLinux eaddb4d449e4 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / 871866e
Default JavaPrivate Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/2/testReport/
Max. process+thread count240 (vs. ulimit of 5500)
modulesC: tez-runtime-internals tez-dag U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/2/console
versionsgit=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

ayushtkn
ayushtkn reviewed Feb 20, 2025
View reviewed changes
Comment threadtez-dag/src/main/java/org/apache/tez/dag/app/DAGAppMaster.java
Comment on lines +2393 to +2395
org.apache.hadoop.ipc.CallerContext.setCurrent(new org.apache.hadoop.ipc.CallerContext
.Builder("tez_appmaster_" + containerId.getApplicationAttemptId()
).build());

@ayushtknayushtknFeb 20, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will overwrite any existing CallerContext, You need to append the values not overwrite

@liangyu-1liangyu-1Feb 21, 2025

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for you review,@ayushtkn
I just fixed this line, is it OK now?

@liangyu-1liangyu-1Feb 21, 2025

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will overwrite any existing CallerContext, You need to append the values not overwrite

But in my opinion, this org.apache.hadoop.ipc.CallerContextis different from org.apache.tez.client.CallerContext, in this main function, it is the first time I set a new value to org.apache.hadoop.ipc.CallerContext, so there will be no existing CallerContext.

is it correct?

@ayushtknayushtknFeb 21, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if any code or application which invokes the DagAppMaster sets it then?

@liangyu-1liangyu-1Feb 21, 2025•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

org.apache.hadoop.ipc.CallerContext is a threadLocal variable, if other application invokes the DagAppMaster, it should start a new thread, and create a new callercontext variable.

@ayushtknayushtknFeb 21, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, then we can live without it.

is there a possibility to extend some tests?

@liangyu-1liangyu-1Feb 21, 2025

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to implement some unit tests, but the ThreadLocal variable could not be accessed. In Hadoop, this ThreadLocal variable is tested by checking the audit log, but I am unable to start a Hadoop Mini Cluster and correctly invoke DAGAppMaster, so I cannot implement this unit test.

However, the code implementation is simple, and there is a similar implementation in this Hadoop pr which is implemented by my coworker, so it should definitely work.

@tez-yetus

tez-yetus commented Feb 21, 2025

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 9sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+0 🆗mvndep2m 2sMaven dependency ordering for branch
+1 💚mvninstall7m 56smaster passed
+1 💚compile0m 49smaster passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚compile0m 43smaster passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚checkstyle0m 51smaster passed
+1 💚javadoc0m 41smaster passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javadoc0m 27smaster passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+0 🆗spotbugs0m 46sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs1m 34smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 7sMaven dependency ordering for patch
+1 💚mvninstall0m 29sthe patch passed
+1 💚compile0m 28sthe patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javac0m 28sthe patch passed
+1 💚compile0m 25sthe patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚javac0m 25sthe patch passed
+1 💚checkstyle0m 18sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 14sthe patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javadoc0m 12sthe patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
-1 ❌findbugs0m 46stez-dag generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
_ Other Tests _
+1 💚unit0m 43stez-runtime-internals in the patch passed.
-1 ❌unit4m 10stez-dag in the patch failed.
+1 💚asflicense0m 18sThe patch does not generate ASF License warnings.
24m 44s
ReasonTests
FindBugsmodule:tez-dag
Dead store to callerContext in org.apache.tez.dag.app.DAGAppMaster.main(String[]) At DAGAppMaster.java:org.apache.tez.dag.app.DAGAppMaster.main(String[]) At DAGAppMaster.java:[line 2393]
Failed junit teststez.dag.app.rm.TestTaskScheduler
SubsystemReport/Notes
DockerClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/3/artifact/out/Dockerfile
GITHUB PR#394
JIRA IssueTEZ-4605
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
unameLinux 54c35e6c85f3 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / 871866e
Default JavaPrivate Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
findbugshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/3/artifact/out/new-findbugs-tez-dag.html
unithttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/3/artifact/out/patch-unit-tez-dag.txt
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/3/testReport/
Max. process+thread count237 (vs. ulimit of 5500)
modulesC: tez-runtime-internals tez-dag U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/3/console
versionsgit=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@tez-yetus

tez-yetus commented Feb 21, 2025

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 10sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+0 🆗mvndep2m 12sMaven dependency ordering for branch
+1 💚mvninstall8m 26smaster passed
+1 💚compile0m 44smaster passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚compile0m 41smaster passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚checkstyle0m 53smaster passed
+1 💚javadoc0m 40smaster passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javadoc0m 30smaster passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+0 🆗spotbugs0m 46sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs1m 37smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 7sMaven dependency ordering for patch
+1 💚mvninstall0m 30sthe patch passed
+1 💚compile0m 27sthe patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javac0m 27sthe patch passed
+1 💚compile0m 24sthe patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚javac0m 24sthe patch passed
+1 💚checkstyle0m 17sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 14sthe patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javadoc0m 14sthe patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚findbugs1m 6sthe patch passed
_ Other Tests _
+1 💚unit0m 43stez-runtime-internals in the patch passed.
+1 💚unit4m 8stez-dag in the patch passed.
+1 💚asflicense0m 18sThe patch does not generate ASF License warnings.
25m 14s
SubsystemReport/Notes
DockerClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/4/artifact/out/Dockerfile
GITHUB PR#394
JIRA IssueTEZ-4605
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
unameLinux 9806e7b044af 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / 871866e
Default JavaPrivate Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/4/testReport/
Max. process+thread count237 (vs. ulimit of 5500)
modulesC: tez-runtime-internals tez-dag U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/4/console
versionsgit=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

ayushtkn
ayushtkn approved these changes Feb 21, 2025
View reviewed changes

@ayushtknayushtkn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tez-yetus

tez-yetus commented Feb 21, 2025

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 19sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚@author0m 1sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+0 🆗mvndep2m 5sMaven dependency ordering for branch
+1 💚mvninstall10m 17smaster passed
+1 💚compile1m 12smaster passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚compile1m 6smaster passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚checkstyle1m 7smaster passed
+1 💚javadoc0m 51smaster passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javadoc0m 43smaster passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+0 🆗spotbugs1m 24sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs2m 28smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for patch
+1 💚mvninstall0m 47sthe patch passed
+1 💚compile0m 53sthe patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javac0m 53sthe patch passed
+1 💚compile0m 45sthe patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚javac0m 45sthe patch passed
+1 💚checkstyle0m 30sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 20sthe patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javadoc0m 19sthe patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
-1 ❌findbugs1m 21stez-dag generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
_ Other Tests _
+1 💚unit0m 55stez-runtime-internals in the patch passed.
+1 💚unit5m 7stez-dag in the patch passed.
+1 💚asflicense0m 20sThe patch does not generate ASF License warnings.
33m 43s
ReasonTests
FindBugsmodule:tez-dag
Dead store to callerContext in org.apache.tez.dag.app.DAGAppMaster.main(String[]) At DAGAppMaster.java:org.apache.tez.dag.app.DAGAppMaster.main(String[]) At DAGAppMaster.java:[line 2393]
SubsystemReport/Notes
DockerClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/5/artifact/out/Dockerfile
GITHUB PR#394
JIRA IssueTEZ-4605
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
unameLinux 27112972b6bd 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / 871866e
Default JavaPrivate Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
findbugshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/5/artifact/out/new-findbugs-tez-dag.html
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/5/testReport/
Max. process+thread count244 (vs. ulimit of 5500)
modulesC: tez-runtime-internals tez-dag U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/5/console
versionsgit=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@tez-yetus

tez-yetus commented Feb 21, 2025

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec27m 53sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+0 🆗mvndep2m 0sMaven dependency ordering for branch
+1 💚mvninstall11m 24smaster passed
+1 💚compile1m 31smaster passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚compile1m 11smaster passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚checkstyle1m 11smaster passed
+1 💚javadoc0m 52smaster passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javadoc0m 43smaster passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+0 🆗spotbugs1m 17sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs2m 19smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 10sMaven dependency ordering for patch
+1 💚mvninstall0m 43sthe patch passed
+1 💚compile0m 48sthe patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javac0m 48sthe patch passed
+1 💚compile0m 41sthe patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚javac0m 41sthe patch passed
+1 💚checkstyle0m 30sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 19sthe patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚javadoc0m 18sthe patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚findbugs1m 52sthe patch passed
_ Other Tests _
+1 💚unit0m 52stez-runtime-internals in the patch passed.
+1 💚unit4m 52stez-dag in the patch passed.
+1 💚asflicense0m 20sThe patch does not generate ASF License warnings.
61m 50s
SubsystemReport/Notes
DockerClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/6/artifact/out/Dockerfile
GITHUB PR#394
JIRA IssueTEZ-4605
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
unameLinux 0f285c39cd82 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / 871866e
Default JavaPrivate Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/6/testReport/
Max. process+thread count239 (vs. ulimit of 5500)
modulesC: tez-runtime-internals tez-dag U: .
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-394/6/console
versionsgit=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@ayushtkn
ayushtkn merged commit e13cab3 into apache:masterFeb 21, 2025
@abstractdogabstractdog changed the title TEZ-4605: Enbale hadoop CallerContext in TezChild and DAGAppMasterTEZ-4605: Enable hadoop CallerContext in TezChild and DAGAppMasterAug 8, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ayushtknayushtknayushtkn approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@liangyu-1@tez-yetus@ayushtkn