HBASE-27843 If moveAndClose fails HFileArchiver should delete any incomplete archive side changes by apurtell · Pull Request #5240 · apache/hbase

apurtell · 2023-05-15T22:07:24Z

When HFiles are placed on a filesystem other than HDFS a rename operation can be a non-atomic file copy operation. It can take a long time to copy a large hfile and if interrupted there may be a partially copied file present at the destination. If we fail to “rename” the files into the archive we will continue to fail indefinitely. Before larger changes are considered, perhaps to StoreFileTracker, we should mitigate this problem.

Apache-HBase · 2023-05-15T23:07:04Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 34s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	5m 50s	master passed
+1 💚	compile	3m 12s	master passed
+1 💚	checkstyle	0m 38s	master passed
+1 💚	spotless	0m 50s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 42s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 14s	the patch passed
+1 💚	compile	2m 58s	the patch passed
+1 💚	javac	2m 58s	the patch passed
+1 💚	checkstyle	0m 36s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	20m 33s	Patch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚	spotless	0m 49s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 50s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		56m 35s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5240
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 3181392950a6 5.4.0-1099-aws #107~18.04.1-Ubuntu SMP Fri Mar 17 16:49:05 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `93ddd70`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	85 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-05-16T02:11:30Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 36s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	5m 52s	master passed
+1 💚	compile	1m 3s	master passed
+1 💚	shadedjars	5m 34s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 35s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 9s	the patch passed
+1 💚	compile	0m 53s	the patch passed
+1 💚	javac	0m 53s	the patch passed
+1 💚	shadedjars	5m 30s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	the patch passed
		_ Other Tests _
+1 💚	unit	211m 21s	hbase-server in the patch passed.
		240m 58s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5240
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux c62d7b445b30 5.4.0-1099-aws #107~18.04.1-Ubuntu SMP Fri Mar 17 16:49:05 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `93ddd70`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/1/testReport/
Max. process+thread count	4654 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-05-16T02:14:53Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 54s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 4s	master passed
+1 💚	compile	0m 39s	master passed
+1 💚	shadedjars	4m 19s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 54s	the patch passed
+1 💚	compile	0m 40s	the patch passed
+1 💚	javac	0m 40s	the patch passed
+1 💚	shadedjars	4m 20s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	the patch passed
		_ Other Tests _
-1 ❌	unit	222m 18s	hbase-server in the patch failed.
		244m 24s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5240
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 30a8ecabb091 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `93ddd70`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/1/testReport/
Max. process+thread count	4437 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

apurtell · 2023-05-16T21:36:04Z

Precommit test failures are not related. The failing tests are related to TLS RPC.

[ERROR] Failures: [ERROR] TestSaslTlsIPC>AbstractTestSecureIPC.testSaslNoCommonQop:284 unexpected exception type thrown; expected:<javax.security.sasl.SaslException> but was:<java.security.cert.CertificateExpiredException> [ERROR] Errors: [ERROR] TestSaslTlsIPC.testDifferentConfWithCryptoAES[4: caKeyType=RSA, certKeyType=RSA, keyPassword=[C@1f760b47, acceptPlainText=true, clientTlsEnabled=true] Â» CertificateExpired NotAfter: Tue May 16 00:00:00 GMT 2023
....

I filed HBASE-27868 for these.

Apache9

A minor nits about comments. Overall LGTM.

…omplete archive side changes When HFiles are placed on a filesystem other than HDFS a rename operation can be a non-atomic file copy operation. It can take a long time to copy a large hfile and if interrupted there may be a partially copied file present at the destination. If we fail to “rename” the files into the archive we will continue to fail indefinitely. Before larger changes are considered, perhaps to StoreFileTracker, we should mitigate this problem.

apurtell · 2023-05-18T22:52:52Z

Addressed review feedback, and rebased.

Apache-HBase · 2023-05-18T23:52:14Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 2s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 35s	master passed
+1 💚	compile	2m 24s	master passed
+1 💚	checkstyle	0m 37s	master passed
+1 💚	spotless	0m 42s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 30s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 23s	the patch passed
+1 💚	compile	2m 33s	the patch passed
+1 💚	javac	2m 33s	the patch passed
+1 💚	checkstyle	0m 52s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	16m 6s	Patch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚	spotless	1m 8s	patch has no errors when running spotless:check.
+1 💚	spotbugs	2m 38s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 16s	The patch does not generate ASF License warnings.
		46m 50s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5240
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 85c6fb65275c 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `71d7996`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	83 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-05-19T02:59:30Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 23s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 25s	master passed
+1 💚	compile	0m 42s	master passed
+1 💚	shadedjars	4m 42s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 31s	the patch passed
+1 💚	compile	0m 42s	the patch passed
+1 💚	javac	0m 42s	the patch passed
+1 💚	shadedjars	4m 40s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 21s	the patch passed
		_ Other Tests _
-1 ❌	unit	208m 50s	hbase-server in the patch failed.
		233m 49s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5240
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux cb21ee8398f9 5.4.0-1099-aws #107~18.04.1-Ubuntu SMP Fri Mar 17 16:49:05 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `71d7996`
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/2/testReport/
Max. process+thread count	4582 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-05-19T03:12:44Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 0s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 29s	master passed
+1 💚	compile	0m 40s	master passed
+1 💚	shadedjars	4m 21s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 49s	the patch passed
+1 💚	compile	0m 41s	the patch passed
+1 💚	javac	0m 41s	the patch passed
+1 💚	shadedjars	4m 24s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	the patch passed
		_ Other Tests _
+1 💚	unit	224m 22s	hbase-server in the patch passed.
		247m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5240
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 1948def04a01 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `71d7996`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/2/testReport/
Max. process+thread count	4129 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5240/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…omplete archive side changes (#5240) When HFiles are placed on a filesystem other than HDFS a rename operation can be a non-atomic file copy operation. It can take a long time to copy a large hfile and if interrupted there may be a partially copied file present at the destination. If we fail to “rename” the files into the archive we will continue to fail indefinitely. Before larger changes are considered, perhaps to StoreFileTracker, we should mitigate this problem. Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Viraj Jasani <vjasani@apache.org> Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>

apurtell requested review from Apache9, virajjasani and wchevreuil May 16, 2023 21:36

Apache9 approved these changes May 17, 2023
View reviewed changes

Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/backup/HFileArchiver.java

sunhelly approved these changes May 17, 2023
View reviewed changes

wchevreuil approved these changes May 17, 2023
View reviewed changes

virajjasani reviewed May 17, 2023
View reviewed changes

Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/backup/HFileArchiver.java Outdated

apurtell added 2 commits May 18, 2023 15:45

Address review feedback
110db1c

apurtell force-pushed the HBASE-27843 branch from 89a88f9 to 110db1cCompare May 18, 2023 22:52

virajjasani approved these changes May 18, 2023
View reviewed changes

apurtell merged commit 7510d06 into apache:masterMay 25, 2023

apurtell deleted the HBASE-27843 branch May 25, 2023 22:42

Navigation Menu

Uh oh!

HBASE-27843 If moveAndClose fails HFileArchiver should delete any incomplete archive side changes - #5240

HBASE-27843 If moveAndClose fails HFileArchiver should delete any incomplete archive side changes#5240
apurtell merged 2 commits into
apache:masterfrom
apurtell:HBASE-27843

apurtell commented May 15, 2023

Uh oh!

Apache-HBase commented May 15, 2023

Uh oh!

Apache-HBase commented May 16, 2023

Uh oh!

Apache-HBase commented May 16, 2023

Uh oh!

apurtell commented May 16, 2023•
edited
Loading

Uh oh!

Apache9 left a comment

Uh oh!

Uh oh!

Uh oh!

apurtell commented May 18, 2023

Uh oh!

Apache-HBase commented May 18, 2023

Uh oh!

Apache-HBase commented May 19, 2023

Uh oh!

Apache-HBase commented May 19, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

apurtell commented May 15, 2023

Uh oh!

Apache-HBase commented May 15, 2023

Uh oh!

Apache-HBase commented May 16, 2023

Uh oh!

Apache-HBase commented May 16, 2023

Uh oh!

apurtell commented May 16, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Apache9 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

apurtell commented May 18, 2023

Uh oh!

Apache-HBase commented May 18, 2023

Uh oh!

Apache-HBase commented May 19, 2023

Uh oh!

Apache-HBase commented May 19, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

apurtell commented May 16, 2023•
edited
Loading