Skip to content

Update hadoop-3.2.4 to hadoop3.3.3 in apache hadoop pom.xml - #6178

Open
weitule wants to merge 1 commit into
apache:branch-2.5from
weitule:branch-2.5
Open

Update hadoop-3.2.4 to hadoop3.3.3 in apache hadoop pom.xml#6178
weitule wants to merge 1 commit into
apache:branch-2.5from
weitule:branch-2.5

Conversation

@weitule

Copy link
Copy Markdown

Fixed the error: org. apache.hadoop.hbase.DoNotRetryIOException: java.lang.UnsatisfiedLinkError: org-apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z Set hbase.table.sanity.checks to false.The error happened when i create table compressed by snappy. The environment based on hadoop 3.3.3 and hbase2.5.10. But hbase-25.10 document said it is compatible with hadoop 3.3.x, I used the configuration: org.apache.hadoop.hbase.io.compress.aircompressor.SnappyCodec,org.apache.hadoop.hbase.io.compress.xerial.SnappyCodec, the error still occurs.So I changed the dependency hadoop3.1.2 to hadoop 3.3.3, the error fixed.
this problem is same as hbase-2.4, to fixed the problem in hbase-2.5.10 is the same way in hbase-2.4

更新对hadoop 版本的依赖
@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 40sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ branch-2.5 Compile Tests _
+1 💚mvninstall2m 56sbranch-2.5 passed
+1 💚compile5m 9sbranch-2.5 passed
+1 💚spotless0m 43sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚mvninstall2m 53sthe patch passed
+1 💚compile5m 13sthe patch passed
+1 💚javac5m 13sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚xmllint0m 0sNo new issues.
+1 💚hadoopcheck20m 5sPatch does not cause any errors with Hadoop 2.10.2 or 3.2.4 3.3.6 3.4.0.
+1 💚spotless0m 43spatch has no errors when running spotless:check.
_ Other Tests _
+1 💚asflicense0m 14sThe patch does not generate ASF License warnings.
40m 0s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6178/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#6178
Optional Testsdupname asflicense javac codespell detsecrets xmllint hadoopcheck spotless compile
unameLinux fd633dfbe208 5.4.0-192-generic #212-Ubuntu SMP Fri Jul 5 09:47:39 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2.5 / 95d8f68
Default JavaEclipse Adoptium-11.0.23+9
Max. process+thread count78 (vs. ulimit of 30000)
modulesC: . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6178/1/console
versionsgit=2.34.1 maven=3.9.8 xmllint=20913
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 42sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2.5 Compile Tests _
+1 💚mvninstall2m 58sbranch-2.5 passed
+1 💚compile2m 6sbranch-2.5 passed
+1 💚javadoc2m 10sbranch-2.5 passed
+1 💚shadedjars5m 5sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall2m 48sthe patch passed
+1 💚compile2m 5sthe patch passed
+1 💚javac2m 5sthe patch passed
+1 💚javadoc2m 6sthe patch passed
+1 💚shadedjars5m 12spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌unit236m 55s/patch-unit-root.txtroot in the patch failed.
267m 15s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6178/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#6178
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 8a2bd96ebc6a 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2.5 / 95d8f68
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6178/1/testReport/
Max. process+thread count4846 (vs. ulimit of 30000)
modulesC: . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6178/1/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 18sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2.5 Compile Tests _
+1 💚mvninstall2m 56sbranch-2.5 passed
+1 💚compile2m 4sbranch-2.5 passed
+1 💚javadoc2m 4sbranch-2.5 passed
+1 💚shadedjars5m 4sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall2m 59sthe patch passed
+1 💚compile2m 3sthe patch passed
+1 💚javac2m 3sthe patch passed
+1 💚javadoc2m 1sthe patch passed
+1 💚shadedjars5m 12spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌unit249m 6s/patch-unit-root.txtroot in the patch failed.
279m 38s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6178/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#6178
Optional Testsjavac javadoc unit shadedjars compile
unameLinux a93dc898e4be 5.4.0-192-generic #212-Ubuntu SMP Fri Jul 5 09:47:39 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2.5 / 95d8f68
Default JavaEclipse Adoptium-11.0.23+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6178/1/testReport/
Max. process+thread count8088 (vs. ulimit of 30000)
modulesC: . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6178/1/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 50sDocker mode activated.
-0 ⚠️yetus0m 5sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2.5 Compile Tests _
+1 💚mvninstall2m 31sbranch-2.5 passed
+1 💚compile1m 48sbranch-2.5 passed
+1 💚javadoc1m 40sbranch-2.5 passed
+1 💚shadedjars4m 46sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚mvninstall2m 18sthe patch passed
+1 💚compile1m 48sthe patch passed
+1 💚javac1m 48sthe patch passed
+1 💚javadoc1m 36sthe patch passed
+1 💚shadedjars4m 41spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌unit292m 44s/patch-unit-root.txtroot in the patch failed.
320m 10s
SubsystemReport/Notes
DockerClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6178/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR#6178
Optional Testsjavac javadoc unit shadedjars compile
unameLinux e43619a835b7 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionbranch-2.5 / 95d8f68
Default JavaTemurin-1.8.0_412-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6178/1/testReport/
Max. process+thread count4601 (vs. ulimit of 30000)
modulesC: . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6178/1/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache9

Copy link
Copy Markdown
Contributor

Could you please provide the jira issue number about which change in hadoop 3.x breaks HBase? We can not simply change hadoop to 3.3.x by default, as there are still users on 3.2.x, we should find other ways to make hbase work with both hadoop versions.

Thanks.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@weitule@Apache-HBase@Apache9