Skip to content

TEZ-4455: Add LoggingHandler in ShuffleHandler pipeline for better debuggability - #249

Merged
abstractdog merged 1 commit into
apache:masterfrom
difin:TEZ-4455
Nov 23, 2022
Merged

TEZ-4455: Add LoggingHandler in ShuffleHandler pipeline for better debuggability#249
abstractdog merged 1 commit into
apache:masterfrom
difin:TEZ-4455

Conversation

@difin

@difindifin commented Nov 22, 2022

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?
Adding LoggingHandler in ShuffleHandler pipeline for better debuggability.

Why are the changes needed?
For corner case debugging, it will be helpful to understand when netty processed OPEN/BOUND/CLOSE/RECEIVED/CONNECTED events along with payload details.
Adding "LoggingHandler" in ChannelPipeline mode to help in debugging.

Does this PR introduce any user-facing change?
No.

How was this patch tested?
Pre-commit testing.

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 28sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+1 💚mvninstall15m 31smaster passed
+1 💚compile0m 42smaster passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚compile0m 40smaster passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚checkstyle1m 11smaster passed
+1 💚javadoc0m 46smaster passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚javadoc0m 36smaster passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+0 🆗spotbugs1m 14sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs1m 11smaster passed
_ Patch Compile Tests _
+1 💚mvninstall0m 27sthe patch passed
+1 💚compile0m 19sthe patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚javac0m 19sthe patch passed
+1 💚compile0m 18sthe patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚javac0m 18sthe patch passed
+1 💚checkstyle0m 11sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 14sthe patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚javadoc0m 13sthe patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚findbugs0m 35sthe patch passed
_ Other Tests _
+1 💚unit2m 51stez-aux-services in the patch passed.
+1 💚asflicense0m 16sThe patch does not generate ASF License warnings.
27m 27s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-249/1/artifact/out/Dockerfile
GITHUB PR#249
JIRA IssueTEZ-4455
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
unameLinux f2e7d93761fd 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitypersonality/tez.sh
git revisionmaster / 8bb27b8
Default JavaPrivate Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Resultshttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-249/1/testReport/
Max. process+thread count1504 (vs. ulimit of 5500)
modulesC: tez-plugins/tez-aux-services U: tez-plugins/tez-aux-services
Console outputhttps://ci-hadoop.apache.org/job/tez-multibranch/job/PR-249/1/console
versionsgit=2.25.1 maven=3.6.3 findbugs=3.0.1
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@abstractdog
abstractdog self-requested a review November 23, 2022 19:24

@abstractdogabstractdog left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@abstractdogabstractdog changed the title TEZ-4455: Add LoggingHandler in ShuffleHandler pipeline for better de…TEZ-4455: Add LoggingHandler in ShuffleHandler pipeline for better debuggabilityNov 23, 2022
@abstractdog
abstractdog merged commit 65f9ee3 into apache:masterNov 23, 2022
prabhjyotsingh pushed a commit to acceldata-io/tez that referenced this pull request Nov 11, 2024
…buggability. (apache#249) (Dmitriy Fingerman reviewed by Laszlo Bodor)
(cherry picked from commit 65f9ee3)
prabhjyotsingh pushed a commit to acceldata-io/tez that referenced this pull request Nov 12, 2024
…buggability. (apache#249) (Dmitriy Fingerman reviewed by Laszlo Bodor)
(cherry picked from commit 65f9ee3)
prabhjyotsingh pushed a commit to acceldata-io/tez that referenced this pull request Nov 12, 2024
…buggability. (apache#249) (Dmitriy Fingerman reviewed by Laszlo Bodor)
(cherry picked from commit 65f9ee3)
(cherry picked from commit 2c2be57)
prabhjyotsingh pushed a commit to acceldata-io/tez that referenced this pull request Nov 12, 2024
…buggability. (apache#249) (Dmitriy Fingerman reviewed by Laszlo Bodor)
(cherry picked from commit 65f9ee3)
(cherry picked from commit 2c2be57)
prabhjyotsingh pushed a commit to acceldata-io/tez that referenced this pull request Nov 12, 2024
…buggability. (apache#249) (Dmitriy Fingerman reviewed by Laszlo Bodor)
(cherry picked from commit 65f9ee3)
(cherry picked from commit 2c2be57)
prabhjyotsingh pushed a commit to acceldata-io/tez that referenced this pull request Nov 12, 2024
…buggability. (apache#249) (Dmitriy Fingerman reviewed by Laszlo Bodor)
(cherry picked from commit 65f9ee3)
(cherry picked from commit 2c2be57)
prabhjyotsingh pushed a commit to acceldata-io/tez that referenced this pull request Nov 12, 2024
…buggability. (apache#249) (Dmitriy Fingerman reviewed by Laszlo Bodor)
(cherry picked from commit 65f9ee3)
(cherry picked from commit 2c2be57)
prabhjyotsingh pushed a commit to acceldata-io/tez that referenced this pull request Nov 20, 2024
…buggability. (apache#249) (Dmitriy Fingerman reviewed by Laszlo Bodor)
(cherry picked from commit 65f9ee3)
(cherry picked from commit 2c2be57)
(cherry picked from commit 12be783)
prabhjyotsingh added a commit to acceldata-io/tez that referenced this pull request Nov 20, 2024
… better debuggability. (apache#249) (Dmitriy Fingerman reviewed by Laszlo Bodor) (#21)
(cherry picked from commit 65f9ee3)
(cherry picked from commit 2c2be57)
(cherry picked from commit 12be783)
Co-authored-by: Dmitriy Fingerman <dmitriy.fingerman@gmail.com>
shubhluck pushed a commit to acceldata-io/tez that referenced this pull request Nov 21, 2024
… better debuggability. (apache#249) (Dmitriy Fingerman reviewed by Laszlo Bodor) (#21)
(cherry picked from commit 65f9ee3)
(cherry picked from commit 2c2be57)
(cherry picked from commit 12be783)
Co-authored-by: Dmitriy Fingerman <dmitriy.fingerman@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@difin@tez-yetus@abstractdog