Skip to content

HBASE-27872 xerial's snappy-java requires GLIBC >= 2.32 - #5245

Merged
apurtell merged 3 commits into
apache:masterfrom
apurtell:HBASE-27872
May 26, 2023
Merged

HBASE-27872 xerial's snappy-java requires GLIBC >= 2.32#5245
apurtell merged 3 commits into
apache:masterfrom
apurtell:HBASE-27872

Conversation

@apurtell

@apurtellapurtell commented May 18, 2023

Copy link
Copy Markdown
Contributor

We need to add a native library load check with a helpful error message if xerial snappy fails to initialize due to a too old glibc or similar reason, and disable the unit test if the native library fails to load.

@apurtell
HBASE-27872 xerial's snappy-java requires GLIBC >= 2.32
b09062f
We need to add a native library load check with a helpful error message
if xerial snappy fails to initialize due to a too old glibc or similar
reason, and disable the unit test if the native library fails to load.
@apurtell

apurtell commented May 18, 2023

Copy link
Copy Markdown
ContributorAuthor

After the proposed changes:

2023-05-18T16:49:25,730 ERROR [Time-limited test {}] xerial.SnappyCodec(58): The Snappy native libraries could not be loaded
java.lang.UnsatisfiedLinkError: /home/apurtell/src/hbase/hbase-compression/hbase-compression-snappy/target/snappy-1.1.9-1a39c2aa-088d-44df-a86b-7bc7d8682d47-libsnappyjava.so:
/lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /home/apurtell/src/hbase/hbase-compression/hbase-compression-snappy/target/snappy-1.1.9-1a39c2aa-088d-44df-a86b-7bc7d8682d47-libsnappyjava.so)
at java.lang.ClassLoader$NativeLibrary.load(Native Method) ~[?:1.8.0_332]
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1950) ~[?:1.8.0_332]
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1832) ~[?:1.8.0_332]
at java.lang.Runtime.load0(Runtime.java:811) ~[?:1.8.0_332]
at java.lang.System.load(System.java:1088) ~[?:1.8.0_332]
at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:180) ~[snappy-java-1.1.9.1.jar:1.1.9.1]
at org.xerial.snappy.SnappyLoader.loadSnappyApi(SnappyLoader.java:155) ~[snappy-java-1.1.9.1.jar:1.1.9.1]
at org.xerial.snappy.Snappy.init(Snappy.java:70) ~[snappy-java-1.1.9.1.jar:1.1.9.1]
at org.xerial.snappy.Snappy.<clinit>(Snappy.java:47) ~[snappy-java-1.1.9.1.jar:1.1.9.1]
at org.apache.hadoop.hbase.io.compress.xerial.SnappyCodec.<clinit>(SnappyCodec.java:55) ~[classes/:?]
at org.apache.hadoop.hbase.io.compress.xerial.TestWALCompressionSnappy.setUpBeforeClass(TestWALCompressionSnappy.java:53) ~[test-classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_332]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_332]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_332]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_332]
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) ~[junit-4.13.2.jar:4.13.2]
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) ~[junit-4.13.2.jar:4.13.2]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_332]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_332] 2023-05-18T16:49:25,736 WARN [Time-limited test {}] xerial.TestWALCompressionSnappy(54): Snappy codec cannot be loaded. Test will not execute.

@Apache-HBase

Apache-HBase commented May 19, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 58sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 5smaster passed
+1 💚compile0m 13smaster passed
+1 💚shadedjars4m 23sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 12smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 49sthe patch passed
+1 💚compile0m 13sthe patch passed
+1 💚javac0m 13sthe patch passed
+1 💚shadedjars4m 22spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 11sthe patch passed
_ Other Tests _
+1 💚unit0m 29shbase-compression-snappy in the patch passed.
17m 59s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#5245
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 05b0d36f7096 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 71d7996
Default JavaTemurin-1.8.0_352-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/1/testReport/
Max. process+thread count241 (vs. ulimit of 30000)
modulesC: hbase-compression/hbase-compression-snappy U: hbase-compression/hbase-compression-snappy
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 19, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 24sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 43smaster passed
+1 💚compile0m 11smaster passed
+1 💚shadedjars4m 48sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 12smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 25sthe patch passed
+1 💚compile0m 11sthe patch passed
+1 💚javac0m 11sthe patch passed
+1 💚shadedjars4m 43spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 10sthe patch passed
_ Other Tests _
+1 💚unit0m 28shbase-compression-snappy in the patch passed.
19m 18s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5245
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 7117cb35d475 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 71d7996
Default JavaEclipse Adoptium-11.0.17+8
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/1/testReport/
Max. process+thread count261 (vs. ulimit of 30000)
modulesC: hbase-compression/hbase-compression-snappy U: hbase-compression/hbase-compression-snappy
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 19, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 17sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall3m 33smaster passed
+1 💚compile0m 17smaster passed
+1 💚checkstyle0m 10smaster passed
+1 💚spotless0m 43sbranch has no errors when running spotless:check.
+1 💚spotbugs0m 22smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 23sthe patch passed
+1 💚compile0m 15sthe patch passed
+1 💚javac0m 15sthe patch passed
+1 💚checkstyle0m 8sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck12m 23sPatch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚spotless0m 41spatch has no errors when running spotless:check.
+1 💚spotbugs0m 26sthe patch passed
_ Other Tests _
+1 💚asflicense0m 10sThe patch does not generate ASF License warnings.
31m 8s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5245
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux 9526a7cbc398 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 71d7996
Default JavaEclipse Adoptium-11.0.17+8
Max. process+thread count84 (vs. ulimit of 30000)
modulesC: hbase-compression/hbase-compression-snappy U: hbase-compression/hbase-compression-snappy
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/1/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 19, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 27sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 15smaster passed
+1 💚compile0m 11smaster passed
+1 💚shadedjars4m 41sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 11smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 51sthe patch passed
+1 💚compile0m 10sthe patch passed
+1 💚javac0m 10sthe patch passed
+1 💚shadedjars4m 39spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 9sthe patch passed
_ Other Tests _
+1 💚unit0m 29shbase-compression-snappy in the patch passed.
17m 50s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#5245
Optional Testsjavac javadoc unit shadedjars compile
unameLinux a6c79b7f6fee 5.4.0-1099-aws #107~18.04.1-Ubuntu SMP Fri Mar 17 16:49:05 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 71d7996
Default JavaTemurin-1.8.0_352-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/2/testReport/
Max. process+thread count242 (vs. ulimit of 30000)
modulesC: hbase-compression/hbase-compression-snappy U: hbase-compression/hbase-compression-snappy
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 19, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 25sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 49smaster passed
+1 💚compile0m 11smaster passed
+1 💚shadedjars4m 44sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 12smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 39sthe patch passed
+1 💚compile0m 11sthe patch passed
+1 💚javac0m 11sthe patch passed
+1 💚shadedjars4m 55spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 10sthe patch passed
_ Other Tests _
+1 💚unit0m 29shbase-compression-snappy in the patch passed.
19m 36s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#5245
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 6e2002e6b171 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 71d7996
Default JavaEclipse Adoptium-11.0.17+8
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/2/testReport/
Max. process+thread count253 (vs. ulimit of 30000)
modulesC: hbase-compression/hbase-compression-snappy U: hbase-compression/hbase-compression-snappy
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented May 19, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 44sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall3m 39smaster passed
+1 💚compile0m 17smaster passed
+1 💚checkstyle0m 9smaster passed
+1 💚spotless0m 42sbranch has no errors when running spotless:check.
+1 💚spotbugs0m 21smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 22sthe patch passed
+1 💚compile0m 16sthe patch passed
+1 💚javac0m 16sthe patch passed
+1 💚checkstyle0m 9sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck12m 25sPatch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚spotless0m 41spatch has no errors when running spotless:check.
+1 💚spotbugs0m 25sthe patch passed
_ Other Tests _
+1 💚asflicense0m 11sThe patch does not generate ASF License warnings.
30m 33s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#5245
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux f82964fb906c 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 71d7996
Default JavaEclipse Adoptium-11.0.17+8
Max. process+thread count83 (vs. ulimit of 30000)
modulesC: hbase-compression/hbase-compression-snappy U: hbase-compression/hbase-compression-snappy
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5245/2/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@apurtell
apurtell merged commit f347867 into apache:masterMay 26, 2023
@apurtell
apurtell deleted the HBASE-27872 branch May 26, 2023 00:06
asfgit pushed a commit that referenced this pull request May 26, 2023
@apurtell
HBASE-27872xerial's snappy-java requires GLIBC >= 2.32 (#5245)
aff6cdb
We need to add a native library load check with a helpful error message
if xerial snappy fails to initialize due to a too old glibc or similar
reason, and disable the unit test if the native library fails to load.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
asfgit pushed a commit that referenced this pull request May 26, 2023
@apurtell
HBASE-27872xerial's snappy-java requires GLIBC >= 2.32 (#5245)
be0ae62
We need to add a native library load check with a helpful error message
if xerial snappy fails to initialize due to a too old glibc or similar
reason, and disable the unit test if the native library fails to load.
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Apache9Apache9Apache9 approved these changes
@virajjasanivirajjasanivirajjasani approved these changes
@sunhellysunhellyAwaiting requested review from sunhelly

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@apurtell@Apache-HBase@Apache9@virajjasani