HDDS-1214. Enable tracing for the datanode read/write path by elek · Pull Request #550 · apache/hadoop

elek · 2019-03-04T14:25:47Z

HDDS-1150 introduced distributed for ozone components. But we have no trace context propagation between the clients and Ozone Datanodes.

As we use Grpc and Ratis on this RPC path the full tracing could be quite complex: we should propagate the trace id in Ratis and include it in all the log entries.

I propose a simplified solution here: to trace only the StateMachine operations.

As Ratis is a library we provide the implementation of the appropriate Raft elements especially the StateMachine and the raft messages. We can add the tracing information to the raft messages (in fact, we already have this field) and we can restore the tracing context during the StateMachine operations.

This approach is very simple (only a few lines of codes) and can show the time of the real write/read operations, but can't see the internals of the Ratis operations.

See: https://issues.apache.org/jira/browse/HDDS-1214

hadoop-yetus · 2019-03-04T15:21:26Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	25	Docker mode activated.
		_ Prechecks _
+1	@author	0	The patch does not contain any @author tags.
-1	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
0	mvndep	25	Maven dependency ordering for branch
+1	mvninstall	988	trunk passed
+1	compile	71	trunk passed
+1	checkstyle	26	trunk passed
+1	mvnsite	72	trunk passed
+1	shadedclient	672	branch has no errors when building and testing our client artifacts.
+1	findbugs	115	trunk passed
+1	javadoc	61	trunk passed
		_ Patch Compile Tests _
0	mvndep	11	Maven dependency ordering for patch
-1	mvninstall	25	container-service in the patch failed.
-1	compile	47	hadoop-hdds in the patch failed.
-1	javac	47	hadoop-hdds in the patch failed.
-0	checkstyle	18	hadoop-hdds: The patch generated 19 new + 0 unchanged - 0 fixed = 19 total (was 0)
-1	mvnsite	23	container-service in the patch failed.
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	703	patch has no errors when building and testing our client artifacts.
-1	findbugs	20	container-service in the patch failed.
+1	javadoc	55	the patch passed
		_ Other Tests _
-1	unit	69	common in the patch failed.
-1	unit	23	container-service in the patch failed.
+1	asflicense	23	The patch does not generate ASF License warnings.
		3255

Reason	Tests
Failed junit tests	hadoop.hdds.security.x509.certificate.client.TestDefaultCertificateClient

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-550/1/artifact/out/Dockerfile
GITHUB PR	#550
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 39b95bcd86a1 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / 15098df
maven	version: Apache Maven 3.3.9
Default Java	1.8.0_191
findbugs	v3.1.0-RC1
mvninstall	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/1/artifact/out/patch-mvninstall-hadoop-hdds_container-service.txt
compile	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/1/artifact/out/patch-compile-hadoop-hdds.txt
javac	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/1/artifact/out/patch-compile-hadoop-hdds.txt
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/1/artifact/out/diff-checkstyle-hadoop-hdds.txt
mvnsite	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/1/artifact/out/patch-mvnsite-hadoop-hdds_container-service.txt
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/1/artifact/out/patch-findbugs-hadoop-hdds_container-service.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/1/artifact/out/patch-unit-hadoop-hdds_common.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/1/artifact/out/patch-unit-hadoop-hdds_container-service.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/1/testReport/
Max. process+thread count	446 (vs. ulimit of 5500)
modules	C: hadoop-hdds/common hadoop-hdds/container-service U: hadoop-hdds
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/1/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-03-05T12:03:13Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	43	Docker mode activated.
		_ Prechecks _
+1	@author	0	The patch does not contain any @author tags.
-1	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
0	mvndep	14	Maven dependency ordering for branch
+1	mvninstall	1074	trunk passed
+1	compile	74	trunk passed
+1	checkstyle	29	trunk passed
+1	mvnsite	70	trunk passed
+1	shadedclient	795	branch has no errors when building and testing our client artifacts.
+1	findbugs	126	trunk passed
+1	javadoc	69	trunk passed
		_ Patch Compile Tests _
0	mvndep	15	Maven dependency ordering for patch
+1	mvninstall	77	the patch passed
+1	compile	74	the patch passed
+1	javac	74	the patch passed
+1	checkstyle	23	the patch passed
+1	mvnsite	67	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	758	patch has no errors when building and testing our client artifacts.
+1	findbugs	134	the patch passed
+1	javadoc	64	the patch passed
		_ Other Tests _
-1	unit	82	common in the patch failed.
+1	unit	66	container-service in the patch passed.
+1	asflicense	32	The patch does not generate ASF License warnings.
		3721

Reason	Tests
Failed junit tests	hadoop.hdds.security.x509.certificate.client.TestDefaultCertificateClient

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-550/2/artifact/out/Dockerfile
GITHUB PR	#550
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux fc16b419e907 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / 0aefe28
maven	version: Apache Maven 3.3.9
Default Java	1.8.0_191
findbugs	v3.1.0-RC1
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/2/artifact/out/patch-unit-hadoop-hdds_common.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/2/testReport/
Max. process+thread count	327 (vs. ulimit of 5500)
modules	C: hadoop-hdds/common hadoop-hdds/container-service U: hadoop-hdds
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/2/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

arp7

+1

…ed by Elek, Marton.

elek · 2019-03-11T14:47:03Z

Thanks @arp7 the review. I rebased the patch, will commit it after a clean jenkins run...

hadoop-yetus · 2019-03-11T16:07:33Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	44	Docker mode activated.
		_ Prechecks _
+1	@author	0	The patch does not contain any @author tags.
-1	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
0	mvndep	48	Maven dependency ordering for branch
+1	mvninstall	1177	trunk passed
+1	compile	79	trunk passed
+1	checkstyle	31	trunk passed
+1	mvnsite	78	trunk passed
+1	shadedclient	823	branch has no errors when building and testing our client artifacts.
+1	findbugs	126	trunk passed
+1	javadoc	72	trunk passed
		_ Patch Compile Tests _
0	mvndep	12	Maven dependency ordering for patch
+1	mvninstall	78	the patch passed
+1	compile	72	the patch passed
+1	javac	72	the patch passed
+1	checkstyle	26	the patch passed
+1	mvnsite	69	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	799	patch has no errors when building and testing our client artifacts.
+1	findbugs	136	the patch passed
+1	javadoc	66	the patch passed
		_ Other Tests _
-1	unit	88	common in the patch failed.
-1	unit	62	container-service in the patch failed.
+1	asflicense	36	The patch does not generate ASF License warnings.
		3952

Reason	Tests
Failed junit tests	hadoop.hdds.security.x509.certificate.client.TestDefaultCertificateClient

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-550/3/artifact/out/Dockerfile
GITHUB PR	#550
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 1c6b6adced10 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `ebb5fa1`
maven	version: Apache Maven 3.3.9
Default Java	1.8.0_191
findbugs	v3.1.0-RC1
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/3/artifact/out/patch-unit-hadoop-hdds_common.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/3/artifact/out/patch-unit-hadoop-hdds_container-service.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/3/testReport/
Max. process+thread count	319 (vs. ulimit of 5500)
modules	C: hadoop-hdds/common hadoop-hdds/container-service U: hadoop-hdds
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-550/3/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

…constructor. vjagadish1989 this has already been reviewed and approved by you and cameronlee314 internally. Please approve here. Thanks! Author: Daniel Nishimura <dnishimura@linkedin.com> Reviewers: Jagadish <jagadish@apache.org> Closesapache#550 from dnishimura/samza-1742-localapplicationrunner-custom-metrics

elek added the ozone label Mar 4, 2019

elek force-pushed the HDDS-1214 branch from e8e29ae to d36b21cCompare March 5, 2019 11:00

arp7 approved these changes Mar 7, 2019
View reviewed changes

HDDS-1214. Enable tracing for the datanode read/write path. Contribut…
99afaa8
…ed by Elek, Marton.

elek force-pushed the HDDS-1214 branch from d36b21c to 99afaa8Compare March 11, 2019 14:44

elek closed this in d17e31eMar 12, 2019

Navigation Menu

Uh oh!

HDDS-1214. Enable tracing for the datanode read/write path - #550

HDDS-1214. Enable tracing for the datanode read/write path#550
elek wants to merge 1 commit into
apache:trunkfrom
elek:HDDS-1214

elek commented Mar 4, 2019

Uh oh!

hadoop-yetus commented Mar 4, 2019

Uh oh!

hadoop-yetus commented Mar 5, 2019

Uh oh!

arp7 left a comment

Uh oh!

elek commented Mar 11, 2019

Uh oh!

hadoop-yetus commented Mar 11, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

elek commented Mar 4, 2019

Uh oh!

hadoop-yetus commented Mar 4, 2019

Uh oh!

hadoop-yetus commented Mar 5, 2019

Uh oh!

arp7 left a comment

Choose a reason for hiding this comment

Uh oh!

elek commented Mar 11, 2019

Uh oh!

hadoop-yetus commented Mar 11, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants