HBASE-26168 Backport HBASE-25811 "The client integration test is fail…

[taklwu]

…ing after HBASE-22120 merged" into branch-2

move opentelemetry jars to client-facing-thirdparty
add opentelemetry jars when init map reduce job dependencies

original commit 8d2a0ef

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 32s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ HBASE-25853 Compile Tests _
+0 🆗	mvndep	0m 17s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 42s	HBASE-25853 passed
+1 💚	compile	1m 14s	HBASE-25853 passed
+1 💚	checkstyle	0m 33s	HBASE-25853 passed
+1 💚	spotbugs	0m 47s	HBASE-25853 passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 21s	the patch passed
+1 💚	compile	1m 13s	the patch passed
+1 💚	javac	1m 13s	the patch passed
+1 💚	checkstyle	0m 30s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 3s	The patch has no ill-formed XML file.
+1 💚	hadoopcheck	11m 59s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotbugs	0m 55s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 26s	The patch does not generate ASF License warnings.
		33m 28s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3634
Optional Tests	dupname asflicense javac xml spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 5d75ec136fa9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-25853 / d807632
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	96 (vs. ulimit of 12500)
modules	C: hbase-mapreduce hbase-assembly U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[taklwu]

this commit is required for the completion of HBASE-25853, otherwise the import TSV nightly job with shaded client artifact fail with NoClassDefFoundError exception. and I tested the nightly tests locally with the following steps (with outputs)

# first maven full build HBASE_SOURCE=/Users/taklwu/workspace/hbase2
cd $HBASE_SOURCE
mvn clean install -DskipTests -Prelease && mvn install -DskipTests site assembly:single -Prelease
# unzip the assembly and hbase-client
cd ~/hbase-nightly install_artifact=$(ls -1 ${HBASE_SOURCE}/hbase-assembly/target/hbase-*-bin.tar.gz | sort | head -n 1)
mkdir hbase-install tar --strip-component=1 -xzf "${install_artifact}" -C "hbase-install"
client_artifact=$(ls -1 ${HBASE_SOURCE}/hbase-assembly/target/hbase-*-bin.tar.gz | sort | tail -n 1)
mkdir hbase-client tar --strip-component=1 -xzf "${client_artifact}" -C "hbase-client"
# run the nightly
rm -rf output-integration/hadoop-2/*
bash $HBASE_SOURCE/dev-support/hbase_nightly_pseudo-distributed-test.sh --single-process --working-dir output-integration/hadoop-2 --hbase-client-install hbase-client hbase-install hadoop-2/bin/hadoop hadoop-2/share/hadoop/yarn/timelineservice hadoop-2/share/hadoop/yarn/test/hadoop-yarn-server-tests-2.10.1-tests.jar hadoop-2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.10.1-tests.jar hadoop-2/bin/mapred | tee ~/hbase-nightly/sanity-check.log
Using HBase client-side artifact
You'll find logs and temp files in /Users/taklwu/hbase-nightly/output-integration/hadoop-2
Hadoop version information:
Hadoop 2.10.1
Subversion https://github.com/apache/hadoop -r 1827467c9a56f133025f28557bfc2c562d78e816
Compiled by centos on 2020-09-14T13:17Z
Compiled with protoc 2.5.0
From source with checksum 3114edef868f1f3824e7d0f68be03650
This command was run using /Users/taklwu/hbase-nightly/hadoop-2/share/hadoop/common/hadoop-common-2.10.1.jar
JAVA_HOME: /Users/taklwu/.sdkman/candidates/java/8.292.10.1-amzn
HBase version information:
HBase 2.5.0-SNAPSHOT
Source code repository git://21992.local/Users/taklwu/workspace/hbase2 revision=723831a9de938faf7c2a1b2e4047b68ed281af52
Compiled by taklwu on Thu Aug 26 14:29:11 PDT 2021
From source with checksum c90bbbd1a0293de7e3621195fad2d2e6ade1a2b82fdcb05ee7690e3fb8b338a743d7428dee3ef18b9c6b6d7de1ea979bc8073f5b9bb686b24fc48aa4e8b48e90
Writing out configuration for HBase.
Starting up Hadoop
waiting for Hadoop to finish starting up.
waiting for Hadoop to finish starting up.
waiting for Hadoop to finish starting up.
Listing HDFS contents
Starting up HBase
running master, logging to /Users/taklwu/hbase-nightly/hbase-install/bin/../logs/hbase-taklwu-master-21992.local.out
retry waiting for hbase to come up.
Setting up table 'test:example' with 1,000 regions
writing out example TSV to example.tsv
uploading example.tsv to HDFS
2021-08-26 14:59:41,249 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2021-08-26 14:59:42,671 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Importing TSV via shaded client artifact for HBase - MapReduce integration.
Verifying row count from import.
Hadoop client jars not given; getting them from 'hadoop classpath' for the example.
Building shaded client example.
Running shaded client example. It'll fetch the set of regions, round-trip them to a file in HDFS, then write them one-per-row into the test table.
Checking on results of example program.
2021-08-26 15:02:29,669 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Verifying row count from example.
Found 1051 rows, which is enough to cover 48 for import, 1000 example's use of user table regions, 2 for example's use of meta/namespace regions, and 1 for example's count record
Shutting down HBase
stopping hbase............
Shutdown: listing HDFS contents
Shutting down Hadoop
# check the command exit code and output of mr-importtsv.err
% echo $?
0
% cat output-integration/hadoop-2/mr-importtsv.err
...
2021-08-26 15:00:00,482 INFO [main] mapreduce.Job: Job job_local361623923_0001 completed successfully
2021-08-26 15:00:00,487 INFO [main] mapreduce.Job: Counters: 21
File System Counters
FILE: Number of bytes read=47342109
FILE: Number of bytes written=48290866
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=756
HDFS: Number of bytes written=0
HDFS: Number of read operations=3
HDFS: Number of large read operations=0
HDFS: Number of write operations=0
Map-Reduce Framework
Map input records=48
Map output records=48
Input split bytes=119
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=0
Total committed heap usage (bytes)=556793856
ImportTsv
Bad Lines=0
File Input Format Counters
Bytes Read=756
File Output Format Counters
Bytes Written=0
...

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 48s	Docker mode activated.
-0 ⚠️	yetus	0m 8s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ HBASE-25853 Compile Tests _
+0 🆗	mvndep	0m 19s	Maven dependency ordering for branch
+1 💚	mvninstall	5m 51s	HBASE-25853 passed
+1 💚	compile	1m 4s	HBASE-25853 passed
+1 💚	shadedjars	10m 23s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 44s	HBASE-25853 passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 23s	Maven dependency ordering for patch
+1 💚	mvninstall	5m 18s	the patch passed
+1 💚	compile	1m 11s	the patch passed
+1 💚	javac	1m 11s	the patch passed
+1 💚	shadedjars	9m 46s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	the patch passed
		_ Other Tests _
+1 💚	unit	10m 33s	hbase-mapreduce in the patch passed.
+1 💚	unit	0m 27s	hbase-assembly in the patch passed.
		49m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3634
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 0e7b949ce26a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-25853 / d807632
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/testReport/
Max. process+thread count	4837 (vs. ulimit of 12500)
modules	C: hbase-mapreduce hbase-assembly U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 40s	Docker mode activated.
-0 ⚠️	yetus	0m 7s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ HBASE-25853 Compile Tests _
+0 🆗	mvndep	0m 21s	Maven dependency ordering for branch
+1 💚	mvninstall	4m 50s	HBASE-25853 passed
+1 💚	compile	0m 59s	HBASE-25853 passed
+1 💚	shadedjars	8m 27s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	HBASE-25853 passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 17s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 44s	the patch passed
+1 💚	compile	1m 2s	the patch passed
+1 💚	javac	1m 2s	the patch passed
+1 💚	shadedjars	8m 40s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 38s	the patch passed
		_ Other Tests _
+1 💚	unit	18m 39s	hbase-mapreduce in the patch passed.
+1 💚	unit	0m 30s	hbase-assembly in the patch passed.
		53m 10s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#3634
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux b3f478963855 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-25853 / d807632
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/testReport/
Max. process+thread count	2616 (vs. ulimit of 12500)
modules	C: hbase-mapreduce hbase-assembly U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3634/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.