HBASE-27927 Skip zero length bytes of WALKey and WALEntry cells to av…

[sunhelly]

…oid replication stuck with WAL compression

[Apache9]

Mind exlaining more about the fix? Reading jira description and the code here, I can not understand what is the problem root cause here...

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 36s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 55s	master passed
+1 💚	compile	2m 29s	master passed
+1 💚	checkstyle	0m 32s	master passed
+1 💚	spotless	0m 43s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 34s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 47s	the patch passed
+1 💚	compile	2m 30s	the patch passed
+1 💚	javac	2m 30s	the patch passed
+1 💚	checkstyle	0m 34s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	10m 17s	Patch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚	spotless	0m 41s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 35s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 9s	The patch does not generate ASF License warnings.
		33m 35s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5301/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5301
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux aa440bff656f 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / cd3f94d
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	80 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5301/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[sunhelly]

Mind exlaining more about the fix? Reading jira description and the code here, I can not understand what is the problem root cause here...

Thanks, @Apache9 .By digging some replication stuck WALs with compression and value compression enabled, we found there exiting zero length for WALKey and cells in WALEntry. I think this may be relevant to the bytebuffer padding codes when compressing, as @apurtell said. Cannot prove it until now, but Compression#createCompressionStream is suspicious, it used BufferedOutputStream, which is a bounded byte buff to flush.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 25s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 48s	master passed
+1 💚	compile	0m 42s	master passed
+1 💚	shadedjars	4m 52s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 32s	the patch passed
+1 💚	compile	0m 42s	the patch passed
+1 💚	javac	0m 42s	the patch passed
+1 💚	shadedjars	4m 48s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	the patch passed
		_ Other Tests _
+1 💚	unit	207m 2s	hbase-server in the patch passed.
		228m 12s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5301/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5301
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 08a3adb142a1 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / cd3f94d
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5301/1/testReport/
Max. process+thread count	3998 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5301/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 32s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 54s	master passed
+1 💚	compile	0m 53s	master passed
+1 💚	shadedjars	5m 12s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 12s	the patch passed
+1 💚	compile	0m 35s	the patch passed
+1 💚	javac	0m 35s	the patch passed
+1 💚	shadedjars	4m 53s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 21s	the patch passed
		_ Other Tests _
+1 💚	unit	223m 40s	hbase-server in the patch passed.
		245m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5301/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5301
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux cf65172062fd 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / cd3f94d
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5301/1/testReport/
Max. process+thread count	4616 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5301/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache9]

Member

Then let's provide a UT to reproduce the problem before merging the code here? It is very strange that why we need to skip lots of zeros in the input stream...