Backport StoreFileTracker (HBASE-26067, HBASE-26584, and others) to branch-2.5 by apurtell · Pull Request #4241 · apache/hbase

apurtell · 2022-03-18T02:22:14Z

In a discussion on dev@ the idea was floated that StoreFileTracker could be backported into branch-2.5 to be released as part of 2.5.0 as an experimental feature.

There are sixteen subtasks on HBASE-26067 and several other tangential commits. These appear to be the core commits:

commit 6aaef89HBASE-26064 Introduce a StoreFileTracker to abstract the store file tracking logic
commit 43b40e9HBASE-25988 Store the store file list by a file (HBASE-25988 Store the store file list by a file #3578)
commit 6e05376HBASE-26079 Use StoreFileTracker when splitting and merging (HBASE-26079 Use StoreFileTracker when splitting and merging #3617)
commit 090b2feHBASE-26224 Introduce a MigrationStoreFileTracker to support migrating from different store file tracker implementations (HBASE-26224 Introduce a MigrationStoreFileTracker to support migratin… #3656)
commit 0ee1689HBASE-26246 Persist the StoreFileTracker configurations to TableDescriptor when creating table (HBASE-26246 Persist the StoreFileTracker configurations to TableDescriptor when creating table #3666)
commit 2052e80HBASE-26248 Should find a suitable way to let users specify the store file tracker implementation (HBASE-26248 Should find a suitable way to let users specify the store… #3665)
commit 5ff0f98HBASE-26264 Add more checks to prevent misconfiguration on store file tracker (HBASE-26264 Add more checks to prevent misconfiguration on store file… #3681)
commit fc4f6d1HBASE-26280 Use store file tracker when snapshoting (HBASE-26280 Use store file tracker when snapshoting #3685)
commit 06db852HBASE-26326 CreateTableProcedure fails when FileBasedStoreFileTracker… (HBASE-26326 CreateTableProcedure fails when FileBasedStoreFileTracker… #3721)
commit e4e7cf8HBASE-26386 Refactor StoreFileTracker implementations to expose the set method (HBASE-26386 Refactor StoreFileTracker implementations to expose the s… #3774)
commit 08d1171HBASE-26328 Clone snapshot doesn't load reference files into FILE SFT impl (HBASE-26328 Clone snapshot doesn't load reference files into FILE SFT impl #3749)
commit 8bec26eHBASE-26263 [Rolling Upgrading] Persist the StoreFileTracker configurations to TableDescriptor for existing tables (HBASE-26263 [Rolling Upgrading] Persist the StoreFileTracker configur… #3700)
commit a288365HBASE-26271 Cleanup the broken store files under data directory (HBASE-26271: Cleanup the broken store files under data directory #3786)
commit d00b5faHBASE-26454 CreateTableProcedure still relies on temp dir and renames… (HBASE-26454 CreateTableProcedure still relies on temp dir and renames… #3845)
commit 771e552HBASE-26286: Add support for specifying store file tracker when restoring or cloning snapshot
commit f16b7b1HBASE-26265 Update ref guide to mention the new store file tracker im… (HBASE-26265 Update ref guide to mention the new store file tracker im… #3942)

And then, from HBASE-26584 and beyond:

commit 755b3b4HBASE-26585 Add SFT configuration to META table descriptor when creating META (HBASE-26585 Add SFT configuration to META table descriptor when creating META #3998)
commit 39c42c7HBASE-26639 The implementation of TestMergesSplitsAddToTracker is problematic (HBASE-26639 The implementation of TestMergesSplitsAddToTracker is pro… #4010)
commit 6e1f5b7HBASE-26586 Should not rely on the global config when setting SFT implementation for a table while upgrading (HBASE-26586 Should not rely on the global config when setting SFT implementation for a table while upgrading #4006)
commit f1dd865HBASE-26654 ModifyTableDescriptorProcedure shoud load TableDescriptor while executing (HBASE-26654 ModifyTableDescriptorProcedure shoud load TableDescriptor… #4034)
commit 8fbc9a2HBASE-26674 Should modify filesCompacting under storeWriteLock (HBASE-26674 Should modify filesCompacting under storeWriteLock #4040)
commit 5aa0fd2HBASE-26675 Data race on Compactor.writer (HBASE-26675 Data race on Compactor.writer #4035)
commit 3021c58HBASE-26700 The way we bypass broken track file is not enough in StoreFileListFile (HBASE-26700 The way we bypass broken track file is not enough in Stor… #4055)
commit a8b68c9HBASE-26690 Modify FSTableDescriptors to not rely on renaming when writing TableDescriptor (HBASE-26690 Modify FSTableDescriptors to not rely on renaming when wr… #4054)
commit dffeb8eHBASE-26587 Introduce a new Admin API to change SFT implementation (HBASE-26587 Introduce a new Admin API to change SFT implementation #4030) (HBASE-26587 Introduce a new Admin API to change SFT implementation (#… #4080)
commit b265fe5HBASE-26673 Implement a shell command for change SFT implementation (HBASE-26673 Implement a shell command for change SFT implementation #4113)
commit 4cdb380HBASE-26640 Reimplement master local region initialization to better work with SFT (HBASE-26640 Reimplement master local region initialization to better … #4111)
commit 77bb153HBASE-26707: Reduce number of renames during bulkload (HBASE-26707: Reduce number of renames during bulkload #4066) (HBASE-26707: Reduce number of renames during bulkload (#4066) #4122)
commit a4b192eHBASE-26611 Changing SFT implementation on disabled table is dangerous (HBASE-26611 Changing SFT implementation on disabled table is dangerous #4082)
commit d3629bbHBASE-26837 Set SFT config when creating TableDescriptor in TestCloneSnapshotProcedure (HBASE-26837 Set SFT config when creating TableDescriptor in TestClone… #4226)

hbase-server tests run locally (Linux aarch64) with -Dsurefire.firstPartForkCount=10 -Dsurefire.secondPartForkCount=5 -Dsurefire.rerunFailingTestsCount=4

First part:

[WARNING] Tests run: 1169, Failures: 0, Errors: 0, Skipped: 5

Second part:

[INFO] Results:
[WARNING] Flakes: [WARNING] org.apache.hadoop.hbase.master.procedure.TestModifyNamespaceProcedure.testRollbackAndDoubleExecution
[ERROR] Run 1: TestModifyNamespaceProcedure.testRollbackAndDoubleExecution:255 expected null, but was:<bar>
[INFO] Run 2: PASS
[WARNING] org.apache.hadoop.hbase.replication.regionserver.TestReplicationSource.testReplicationSourceInitializingMetric
[ERROR] Run 1: TestReplicationSource.testReplicationSourceInitializingMetric:583 Waiting timed out after [1,000] msec
[INFO] Run 2: PASS
[WARNING] Tests run: 2255, Failures: 0, Errors: 0, Skipped: 15, Flakes: 2

TestReplicationSource is a known flake.

TestModifyNamespaceProcedure may be a new flake, but most likely a test issue, because the failure does not reproduce when the test is run by itself. I have a script that runs tests in a loop and after 100 repetitions there was no error:

$ ~/bin/test-to-fail -n 100 TestModifyNamespaceProcedure
...
NO FAILURES

apurtell · 2022-03-18T02:25:23Z

This isn't meant for code review. The description on this issue lists all commit SHAs and PR#s for the individual changes that were cherry picked from branch-2. If you want to have a look, it is all there, and the PR branch contains all respective commits in order. A sanity check on inclusion on all necessary changes would be helpful.

The result should test out well and be ready for merge. If there are test issues this is the venue for addressing them.

Apache-HBase · 2022-03-18T03:13:17Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 33s	Docker mode activated.
-0 ⚠️	yetus	0m 5s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.5 Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 10s	branch-2.5 passed
+1 💚	compile	1m 28s	branch-2.5 passed
+1 💚	shadedjars	0m 21s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	3m 1s	branch-2.5 passed
-0 ⚠️	patch	4m 8s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 1s	the patch passed
+1 💚	compile	1m 30s	the patch passed
+1 💚	javac	1m 30s	the patch passed
+1 💚	shadedjars	0m 20s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 23s	hbase-server generated 3 new + 21 unchanged - 0 fixed = 24 total (was 21)
-0 ⚠️	javadoc	1m 24s	root generated 3 new + 56 unchanged - 0 fixed = 59 total (was 56)
		_ Other Tests _
-1 ❌	unit	21m 10s	root in the patch failed.
		38m 4s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#4241
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a7e202aed49d 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `2e8f0a0`
Default Java	AdoptOpenJDK-1.8.0_282-b08
javadoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/artifact/yetus-jdk8-hadoop2-check/output/diff-javadoc-javadoc-hbase-server.txt
javadoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/artifact/yetus-jdk8-hadoop2-check/output/diff-javadoc-javadoc-root.txt
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/testReport/
Max. process+thread count	674 (vs. ulimit of 12500)
modules	C: hbase-protocol-shaded hbase-client hbase-server hbase-mapreduce hbase-thrift hbase-shell hbase-it . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-03-18T03:54:56Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 58s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 1s	No case conflicting files found.
+0 🆗	prototool	0m 0s	prototool was not available.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2.5 Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 44s	branch-2.5 passed
+1 💚	compile	7m 27s	branch-2.5 passed
+1 💚	checkstyle	1m 28s	branch-2.5 passed
+0 🆗	refguide	2m 26s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	spotbugs	13m 50s	branch-2.5 passed
-0 ⚠️	patch	1m 41s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 21s	the patch passed
-1 ❌	compile	5m 11s	root in the patch failed.
-0 ⚠️	cc	5m 11s	root in the patch failed.
-0 ⚠️	javac	5m 11s	root in the patch failed.
-0 ⚠️	checkstyle	1m 28s	root: The patch generated 21 new + 930 unchanged - 48 fixed = 951 total (was 978)
-0 ⚠️	rubocop	0m 19s	The patch generated 42 new + 447 unchanged - 1 fixed = 489 total (was 448)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+0 🆗	refguide	1m 51s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	hadoopcheck	11m 25s	Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1.
+1 💚	hbaseprotoc	4m 59s	the patch passed
+1 💚	spotbugs	14m 42s	the patch passed
		_ Other Tests _
+1 💚	asflicense	1m 11s	The patch does not generate ASF License warnings.
		79m 19s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4241
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop refguide
uname	Linux 11e0316f836c 5.4.0-1043-aws #45~18.04.1-Ubuntu SMP Fri Apr 9 23:32:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `2e8f0a0`
Default Java	AdoptOpenJDK-1.8.0_282-b08
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4241/1/yetus-general-check/output/branch-site/book.html
compile	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/artifact/yetus-general-check/output/patch-compile-root.txt
cc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/artifact/yetus-general-check/output/patch-compile-root.txt
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/artifact/yetus-general-check/output/patch-compile-root.txt
checkstyle	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/artifact/yetus-general-check/output/diff-checkstyle-root.txt
rubocop	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/artifact/yetus-general-check/output/diff-patch-rubocop.txt
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4241/1/yetus-general-check/output/patch-site/book.html
Max. process+thread count	140 (vs. ulimit of 12500)
modules	C: hbase-protocol-shaded hbase-client hbase-server hbase-mapreduce hbase-thrift hbase-shell hbase-it . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2 rubocop=0.80.0
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-03-18T06:28:54Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 53s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.5 Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 39s	branch-2.5 passed
+1 💚	compile	1m 35s	branch-2.5 passed
+1 💚	shadedjars	0m 19s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	3m 20s	branch-2.5 passed
-0 ⚠️	patch	4m 18s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 19s	the patch passed
+1 💚	compile	1m 37s	the patch passed
+1 💚	javac	1m 37s	the patch passed
+1 💚	shadedjars	0m 18s	patch has no errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 21s	hbase-server generated 9 new + 88 unchanged - 2 fixed = 97 total (was 90)
-0 ⚠️	javadoc	1m 39s	root generated 9 new + 209 unchanged - 2 fixed = 218 total (was 211)
		_ Other Tests _
+1 💚	unit	212m 1s	root in the patch passed.
		233m 36s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4241
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 4490b4a6ee52 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `2e8f0a0`
Default Java	AdoptOpenJDK-11.0.10+9
javadoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/artifact/yetus-jdk11-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt
javadoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/artifact/yetus-jdk11-hadoop3-check/output/diff-javadoc-javadoc-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/testReport/
Max. process+thread count	5017 (vs. ulimit of 12500)
modules	C: hbase-protocol-shaded hbase-client hbase-server hbase-mapreduce hbase-thrift hbase-shell hbase-it . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

apurtell · 2022-03-18T17:28:50Z

Latest precommit result is good:

#4241 (comment)

There is an error prone result to address:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile (default-testCompile) on project hbase-server: Compilation failure
[ERROR] /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4241@2/yetus-general-check/src/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMergesSplitsAddToTracker.java:[237,36] error: [ReturnValueIgnored] Return value of 'contains' must be used
[ERROR] (see https://errorprone.info/bugpattern/ReturnValueIgnored)

Not strictly necessary to address javadoc or checkstyle issues in a backport, I think, but let me do it anyway.

I will also see if I can reproduce the TestHFileProcedurePrettyPrinter failure on the earlier precommit run. Perhaps it is flaky now.

apurtell · 2022-03-19T00:53:46Z

hbase-server tests run locally (Linux aarch64) with -Dsurefire.firstPartForkCount=10 -Dsurefire.secondPartForkCount=5 -Dsurefire.rerunFailingTestsCount=4

First part:

[WARNING] Tests run: 1169, Failures: 0, Errors: 0, Skipped: 5

Second part:

[INFO] Results:
[WARNING] Flakes: [WARNING] org.apache.hadoop.hbase.master.procedure.TestModifyNamespaceProcedure.testRollbackAndDoubleExecution
[ERROR] Run 1: TestModifyNamespaceProcedure.testRollbackAndDoubleExecution:255 expected null, but was:<bar>
[INFO] Run 2: PASS
[WARNING] org.apache.hadoop.hbase.replication.regionserver.TestReplicationSource.testReplicationSourceInitializingMetric
[ERROR] Run 1: TestReplicationSource.testReplicationSourceInitializingMetric:583 Waiting timed out after [1,000] msec
[INFO] Run 2: PASS
[WARNING] Tests run: 2255, Failures: 0, Errors: 0, Skipped: 15, Flakes: 2

TestReplicationSource is a known flake.

TestModifyNamespaceProcedure may be a new flake, but most likely a test issue, because the failure does not reproduce when the test is run by itself. I have a script that runs tests in a loop and after 100 repetitions there was no error:

$ ~/bin/test-to-fail -n 100 TestModifyNamespaceProcedure
...
NO FAILURES

Regarding the TestHFileProcedurePrettyPrinter failure on an above precommit report, I cannot reproduce it. I have a script that runs tests in a loop and after 100 repetitions there was no error:

$ ~/bin/test-to-fail -n 100 TestHFileProcedurePrettyPrinter
...
NO FAILURES

apurtell · 2022-03-19T00:54:54Z

Rebase on HEAD of branch-2.5 and fix errorprone, checkstyle, and javadoc findings.

Apache-HBase · 2022-03-19T01:35:53Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 41s	Docker mode activated.
-0 ⚠️	yetus	0m 5s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.5 Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 31s	branch-2.5 passed
+1 💚	compile	1m 51s	branch-2.5 passed
+1 💚	shadedjars	0m 22s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	3m 15s	branch-2.5 passed
-0 ⚠️	patch	4m 20s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 8s	the patch passed
+1 💚	compile	1m 32s	the patch passed
+1 💚	javac	1m 32s	the patch passed
+1 💚	shadedjars	0m 20s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	3m 29s	the patch passed
		_ Other Tests _
-1 ❌	unit	21m 20s	root in the patch failed.
		39m 52s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#4241
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux ce1b78d4e890 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `b11b18b`
Default Java	AdoptOpenJDK-1.8.0_282-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/2/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/2/testReport/
Max. process+thread count	662 (vs. ulimit of 12500)
modules	C: hbase-protocol-shaded hbase-client hbase-server hbase-mapreduce hbase-thrift hbase-shell hbase-it . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-03-19T02:17:35Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 47s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 1s	No case conflicting files found.
+0 🆗	prototool	0m 0s	prototool was not available.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2.5 Compile Tests _
+0 🆗	mvndep	0m 15s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 48s	branch-2.5 passed
+1 💚	compile	7m 29s	branch-2.5 passed
+1 💚	checkstyle	1m 27s	branch-2.5 passed
+0 🆗	refguide	2m 17s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	spotbugs	13m 44s	branch-2.5 passed
-0 ⚠️	patch	1m 40s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 24s	the patch passed
+1 💚	compile	7m 29s	the patch passed
+1 💚	cc	7m 29s	the patch passed
-0 ⚠️	javac	7m 29s	root generated 1 new + 1666 unchanged - 1 fixed = 1667 total (was 1667)
-0 ⚠️	checkstyle	1m 26s	root: The patch generated 5 new + 937 unchanged - 48 fixed = 942 total (was 985)
-0 ⚠️	rubocop	0m 13s	The patch generated 42 new + 447 unchanged - 1 fixed = 489 total (was 448)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+0 🆗	refguide	1m 50s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	hadoopcheck	11m 24s	Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1.
+1 💚	hbaseprotoc	5m 1s	the patch passed
+1 💚	spotbugs	14m 41s	the patch passed
		_ Other Tests _
+1 💚	asflicense	1m 8s	The patch does not generate ASF License warnings.
		81m 17s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4241
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop refguide
uname	Linux 17defd929c77 5.4.0-1043-aws #45~18.04.1-Ubuntu SMP Fri Apr 9 23:32:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `b11b18b`
Default Java	AdoptOpenJDK-1.8.0_282-b08
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4241/2/yetus-general-check/output/branch-site/book.html
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/2/artifact/yetus-general-check/output/diff-compile-javac-root.txt
checkstyle	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/2/artifact/yetus-general-check/output/diff-checkstyle-root.txt
rubocop	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/2/artifact/yetus-general-check/output/diff-patch-rubocop.txt
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4241/2/yetus-general-check/output/patch-site/book.html
Max. process+thread count	138 (vs. ulimit of 12500)
modules	C: hbase-protocol-shaded hbase-client hbase-server hbase-mapreduce hbase-thrift hbase-shell hbase-it . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/2/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2 rubocop=0.80.0
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-03-19T04:49:04Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 48s	Docker mode activated.
-0 ⚠️	yetus	0m 6s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.5 Compile Tests _
+0 🆗	mvndep	0m 34s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 19s	branch-2.5 passed
+1 💚	compile	1m 36s	branch-2.5 passed
+1 💚	shadedjars	0m 19s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	3m 20s	branch-2.5 passed
-0 ⚠️	patch	4m 23s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 9s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 13s	the patch passed
+1 💚	compile	1m 36s	the patch passed
+1 💚	javac	1m 36s	the patch passed
+1 💚	shadedjars	0m 19s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	3m 20s	the patch passed
		_ Other Tests _
+1 💚	unit	212m 25s	root in the patch passed.
		232m 52s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4241
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux f227447522d0 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `b11b18b`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/2/testReport/
Max. process+thread count	4960 (vs. ulimit of 12500)
modules	C: hbase-protocol-shaded hbase-client hbase-server hbase-mapreduce hbase-thrift hbase-shell hbase-it . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

wchevreuil · 2022-03-22T12:26:22Z

Thanks @apurtell ! Just noticed this TestHFileProcedurePrettyPrinter consistently fails to me locally, when running it individually. It passes on branch-2.5, though. I suspect there might be some regression here, let me check it further and get back here once I have some findings.

wchevreuil · 2022-03-22T13:31:14Z

Quick update: Looks like it was introduced by HBASE-26640.

joshelser · 2022-03-23T02:03:15Z

I'll try to give this a one-over tomorrow, but big +1 on merging this when QA comes back green. (just in case life gets the better of me)

wchevreuil

I believe the TestHFileProcedurePrettyPrinter failures are legit. I think this test is also failing on branch-2. From my analysis, this was introduced on branch-2 after we backported HBASE-26640 on branch-2. It doesn't affect master branch because it contains the changes from HBASE-25368, which for some reason, wasn't backported to any base-2 branches.

I had created HBASE-26881 PR for backporting HBASE-25368 into branch-2. With those changes, the test is now passing. Let's include that one here once it goes to branch-2.

apurtell · 2022-03-25T04:09:17Z

I had created #4267 PR for backporting HBASE-25368 into branch-2. With those changes, the test is now passing. Let's include that one here once it goes to branch-2.

Sounds good. I approved #4267 . I'll merge it tomorrow if it has not been merged yet, then will pick it to this PR.

wchevreuil · 2022-03-25T10:08:03Z

I had created #4267 PR for backporting HBASE-25368 into branch-2. With those changes, the test is now passing. Let's include that one here once it goes to branch-2.
Sounds good. I approved #4267 . I'll merge it tomorrow if it has not been merged yet, then will pick it to this PR.

Thanks, Andrew. Had just merged that backport PR.

apurtell · 2022-03-25T15:39:26Z

HBASE-26881 has been cherry picked to the PR branch and the branch was rebased on branch-2.5 and force pushed just now as fb739b1.

I take this comment from @joshelser :

I'll try to give this a one-over tomorrow, but big +1 on merging this when QA comes back green. (just in case life gets the better of me)

and the last precommit result here as approval to merge but will wait at least 24 hours for any additional review or comment, and for precommit testing of the force push to complete.

apurtell · 2022-03-25T15:41:08Z

Thanks so much for digging in on the TestHFileProcedurePrettyPrinter failure here @wchevreuil

Apache-HBase · 2022-03-25T16:54:44Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 26s	Docker mode activated.
-0 ⚠️	yetus	0m 8s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.5 Compile Tests _
+0 🆗	mvndep	0m 15s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 51s	branch-2.5 passed
+1 💚	compile	2m 40s	branch-2.5 passed
+1 💚	shadedjars	0m 28s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	5m 56s	branch-2.5 passed
-0 ⚠️	patch	7m 27s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s	Maven dependency ordering for patch
+1 💚	mvninstall	4m 21s	the patch passed
+1 💚	compile	3m 0s	the patch passed
+1 💚	javac	3m 0s	the patch passed
+1 💚	shadedjars	0m 30s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	6m 18s	the patch passed
		_ Other Tests _
-1 ❌	unit	42m 10s	root in the patch failed.
		74m 0s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4241
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux afc0feba077b 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `1404539`
Default Java	AdoptOpenJDK-11.0.10+9
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/3/testReport/
Max. process+thread count	694 (vs. ulimit of 12500)
modules	C: hbase-protocol-shaded hbase-client hbase-server hbase-mapreduce hbase-thrift hbase-shell hbase-it . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/3/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

wchevreuil · 2022-03-25T21:11:58Z

I believe the latest test failure is unrelated, got it passing locally.

…racking logic Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Conflicts: hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreFlusher.java hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultStoreEngine.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScannerClosure.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStripeStoreEngine.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestDateTieredCompactor.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactor.java

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

) Signed-off-by: Duo Zhang <zhangduo@apache.org> Conflicts: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java

… impl (apache#3749) Signed-off-by: Duo Zhang <zhangduo@apache.org>

…ations to TableDescriptor for existing tables (apache#3700) Signed-off-by: Duo Zhang <zhangduo@apache.org> Reviewed-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

…he#3786) Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Josh Elser <elserj@apache.org> Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org> Conflicts: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java

apache#3845) Signed-off-by: Duo Zhang <zhangduo@apache.org>

…ring or cloning snapshot Closesapache#3851 Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Josh Elser <elserj@apache.org>

apache#3942)

…ing META (apache#3998) Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Josh Elser <elserj@apache.org> (cherry picked from commit baeb51f)

…blematic (apache#4010) Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org> Conflicts: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/storefiletracker/TestStoreFileTracker.java

…lementation for a table while upgrading (apache#4006) Signed-off-by: GeorryHuang <huangzhuoyue@apache.org> Signed-off-by: Josh Elser <elserj@apache.org> Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

… while executing (apache#4034) Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>

…e#4040) Signed-off-by: Josh Elser <elserj@apache.org>

Signed-off-by: Xin Sun <ddupgs@gmail.com>

…eFileListFile (apache#4055) Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

…iting TableDescriptor (apache#4054) Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

…pache#4030) (apache#4080) Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org> Reviewed-by: Josh Elser <elserj@apache.org>

…pache#4113) Signed-off-by: Duo Zhang <zhangduo@apache.org>

…work with SFT (apache#4111) Signed-off-by: Josh Elser <elserj@apache.org> Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

…pache#4122) Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org> Conflicts: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBulkloadBase.java

apache#4082) Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>

…SnapshotProcedure (apache#4226) Signed-off-by: Xiaolin Ha <haxiaolin@apache.org> (cherry picked from commit 5b28d76)

Signed-off-by: Andrew Purtell <apurtell@apache.org>

…others) to branch-2.5 Previous cherry picks: commit 6aaef89 HBASE-26064 Introduce a StoreFileTracker to abstract the store file tracking logic commit 43b40e9 HBASE-25988 Store the store file list by a file apache#3578) commit 6e05376 HBASE-26079 Use StoreFileTracker when splitting and merging apache#3617) commit 090b2fe HBASE-26224 HBASE-26224 Introduce a MigrationStoreFileTracker to support migratin… apache#3656) commit 0ee1689 HBASE-26246 Persist the StoreFileTracker configurations to TableDescriptor when creating table apache#3666) commit 2052e80 HBASE-26248 Should find a suitable way to let users specify the store… apache#3665) commit 5ff0f98 HBASE-26264 Add more checks to prevent misconfiguration on store file… apache#3681) commit fc4f6d1 HBASE-26280 HBASE-26280 Use store file tracker when snapshoting apache#3685) commit 06db852 HBASE-26326 CreateTableProcedure fails when FileBasedStoreFileTracker… apache#3721) commit e4e7cf8 HBASE-26386 Refactor StoreFileTracker implementations to expose the s… apache#3774) commit 08d1171 HBASE-26328 Clone snapshot doesn't load reference files into FILE SFT impl apache#3749) commit 8bec26e HBASE-26263 [Rolling Upgrading] Persist the StoreFileTracker configur… apache#3700) commit a288365 HBASE-26271: Cleanup the broken store files under data directory apache#3786) commit d00b5fa HBASE-26454 CreateTableProcedure still relies on temp dir and renames… apache#3845) commit 771e552 HBASE-26286: Add support for specifying store file tracker when restoring or cloning snapshot commit f16b7b1 HBASE-26265 Update ref guide to mention the new store file tracker im… apache#3942) commit 755b3b4 HBASE-26585 Add SFT configuration to META table descriptor when creating META apache#3998) commit 39c42c7 HBASE-26639 The implementation of TestMergesSplitsAddToTracker is pro… apache#4010) commit 6e1f5b7 HBASE-26586 Should not rely on the global config when setting SFT implementation for a table while upgrading apache#4006) commit f1dd865 HBASE-26654 ModifyTableDescriptorProcedure shoud load TableDescriptor… apache#4034) commit 8fbc9a2 HBASE-26674 Should modify filesCompacting under storeWriteLock apache#4040) commit 5aa0fd2 HBASE-26675 Data race on Compactor.writer apache#4035) commit 3021c58 HBASE-26700 The way we bypass broken track file is not enough in Stor… apache#4055) commit a8b68c9 HBASE-26690 Modify FSTableDescriptors to not rely on renaming when wr… apache#4054) commit dffeb8e HBASE-26587 Introduce a new Admin API to change SFT implementation (#… apache#4080) commit b265fe5 HBASE-26673 Implement a shell command for change SFT implementation apache#4113) commit 4cdb380 HBASE-26640 Reimplement master local region initialization to better … apache#4111) commit 77bb153 HBASE-26707: Reduce number of renames during bulkload (apache#4066) apache#4122) commit a4b192e HBASE-26611 Changing SFT implementation on disabled table is dangerous apache#4082) commit d3629bb HBASE-26837 Set SFT config when creating TableDescriptor in TestClone… apache#4226) commit 541d748 HBASE-26881 Backport HBASE-25368 to branch-2 (apache#4267) Fixups for precommit error prone, checkstyle, and javadoc warnings after applying cherry picks. Signed-off-by: Josh Elser <elserj@apache.org> Reviewed-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

Apache-HBase · 2022-03-26T17:23:59Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 57s	Docker mode activated.
-0 ⚠️	yetus	0m 6s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.5 Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for branch
-1 ❌	mvninstall	3m 11s	root in branch-2.5 failed.
+1 💚	compile	2m 29s	branch-2.5 passed
-1 ❌	shadedjars	4m 52s	branch has 13 errors when building our shaded downstream artifacts.
+1 💚	javadoc	3m 47s	branch-2.5 passed
-0 ⚠️	patch	9m 24s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 9s	Maven dependency ordering for patch
-1 ❌	mvninstall	2m 39s	root in the patch failed.
+1 💚	compile	1m 59s	the patch passed
+1 💚	javac	1m 59s	the patch passed
-1 ❌	shadedjars	4m 58s	patch has 13 errors when building our shaded downstream artifacts.
+1 💚	javadoc	4m 35s	the patch passed
		_ Other Tests _
-1 ❌	unit	30m 18s	root in the patch failed.
		63m 17s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#4241
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux c5fff76d8bf2 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `64a2e9c`
Default Java	AdoptOpenJDK-1.8.0_282-b08
mvninstall	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-jdk8-hadoop2-check/output/branch-mvninstall-root.txt
shadedjars	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-jdk8-hadoop2-check/output/branch-shadedjars.txt
mvninstall	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-jdk8-hadoop2-check/output/patch-mvninstall-root.txt
shadedjars	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-jdk8-hadoop2-check/output/patch-shadedjars.txt
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/testReport/
Max. process+thread count	909 (vs. ulimit of 12500)
modules	C: hbase-protocol-shaded hbase-client hbase-server hbase-mapreduce hbase-thrift hbase-shell hbase-it . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-03-26T17:47:21Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 58s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 1s	No case conflicting files found.
+0 🆗	prototool	0m 0s	prototool was not available.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2.5 Compile Tests _
+0 🆗	mvndep	0m 14s	Maven dependency ordering for branch
-1 ❌	mvninstall	3m 5s	root in branch-2.5 failed.
+1 💚	compile	8m 30s	branch-2.5 passed
+1 💚	checkstyle	1m 51s	branch-2.5 passed
+0 🆗	refguide	2m 54s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	spotbugs	16m 5s	branch-2.5 passed
-0 ⚠️	patch	10m 26s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 9s	Maven dependency ordering for patch
-1 ❌	mvninstall	2m 39s	root in the patch failed.
+1 💚	compile	9m 54s	the patch passed
+1 💚	cc	9m 54s	the patch passed
-0 ⚠️	javac	9m 54s	root generated 5 new + 1665 unchanged - 5 fixed = 1670 total (was 1670)
-0 ⚠️	checkstyle	1m 48s	root: The patch generated 5 new + 967 unchanged - 48 fixed = 972 total (was 1015)
-0 ⚠️	rubocop	0m 16s	The patch generated 42 new + 447 unchanged - 1 fixed = 489 total (was 448)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+0 🆗	refguide	2m 36s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
-1 ❌	hadoopcheck	4m 54s	The patch causes 13 errors with Hadoop v2.10.0.
-1 ❌	hadoopcheck	9m 24s	The patch causes 13 errors with Hadoop v3.1.2.
-1 ❌	hadoopcheck	13m 6s	The patch causes 13 errors with Hadoop v3.2.1.
-1 ❌	hbaseprotoc	1m 11s	root in the patch failed.
+1 💚	spotbugs	12m 33s	the patch passed
		_ Other Tests _
+1 💚	asflicense	1m 13s	The patch does not generate ASF License warnings.
		86m 37s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4241
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop refguide
uname	Linux 00cbfd44e178 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `64a2e9c`
Default Java	AdoptOpenJDK-1.8.0_282-b08
mvninstall	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-general-check/output/branch-mvninstall-root.txt
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4241/4/yetus-general-check/output/branch-site/book.html
mvninstall	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-general-check/output/patch-mvninstall-root.txt
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-general-check/output/diff-compile-javac-root.txt
checkstyle	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-general-check/output/diff-checkstyle-root.txt
rubocop	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-general-check/output/diff-patch-rubocop.txt
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-4241/4/yetus-general-check/output/patch-site/book.html
hadoopcheck	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-general-check/output/patch-javac-2.10.0.txt
hadoopcheck	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-general-check/output/patch-javac-3.1.2.txt
hadoopcheck	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-general-check/output/patch-javac-3.2.1.txt
hbaseprotoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-general-check/output/patch-hbaseprotoc-root.txt
Max. process+thread count	139 (vs. ulimit of 12500)
modules	C: hbase-protocol-shaded hbase-client hbase-server hbase-mapreduce hbase-thrift hbase-shell hbase-it . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2 rubocop=0.80.0
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-03-26T20:26:02Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 44s	Docker mode activated.
-0 ⚠️	yetus	0m 5s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.5 Compile Tests _
+0 🆗	mvndep	0m 50s	Maven dependency ordering for branch
-1 ❌	mvninstall	2m 52s	root in branch-2.5 failed.
+1 💚	compile	1m 40s	branch-2.5 passed
-1 ❌	shadedjars	3m 56s	branch has 13 errors when building our shaded downstream artifacts.
+1 💚	javadoc	3m 25s	branch-2.5 passed
-0 ⚠️	patch	8m 7s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
-1 ❌	mvninstall	2m 23s	root in the patch failed.
+1 💚	compile	1m 39s	the patch passed
+1 💚	javac	1m 39s	the patch passed
-1 ❌	shadedjars	3m 55s	patch has 13 errors when building our shaded downstream artifacts.
+1 💚	javadoc	3m 23s	the patch passed
		_ Other Tests _
+1 💚	unit	215m 40s	root in the patch passed.
		245m 16s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4241
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 0dc0a28dd861 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.5 / `64a2e9c`
Default Java	AdoptOpenJDK-11.0.10+9
mvninstall	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-jdk11-hadoop3-check/output/branch-mvninstall-root.txt
shadedjars	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt
mvninstall	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-jdk11-hadoop3-check/output/patch-mvninstall-root.txt
shadedjars	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/testReport/
Max. process+thread count	5006 (vs. ulimit of 12500)
modules	C: hbase-protocol-shaded hbase-client hbase-server hbase-mapreduce hbase-thrift hbase-shell hbase-it . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4241/4/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

apurtell requested review from Apache9, joshelser and wchevreuil March 18, 2022 02:22

apurtell force-pushed the HBASE-26826-2.5 branch from 4e124be to 2d34334Compare March 19, 2022 00:54

apurtell requested a review from virajjasani March 19, 2022 00:57

wchevreuil requested changes Mar 23, 2022
View reviewed changes

apurtell force-pushed the HBASE-26826-2.5 branch from 2d34334 to fb739b1Compare March 25, 2022 15:39

apurtell requested a review from wchevreuil March 25, 2022 15:40

Apache9and others added 3 commits March 26, 2022 09:15

HBASE-25988 Store the store file list by a file (apache#3578)
177cbf9
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

HBASE-26079 Use StoreFileTracker when splitting and merging (apache#3617
37e0b0e
) Signed-off-by: Duo Zhang <zhangduo@apache.org> Conflicts: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHStoreFile.java

wchevreuiland others added 21 commits March 26, 2022 09:15

HBASE-26328 Clone snapshot doesn't load reference files into FILE SFT…
73c0633
… impl (apache#3749) Signed-off-by: Duo Zhang <zhangduo@apache.org>

HBASE-26263 [Rolling Upgrading] Persist the StoreFileTracker configur…
d6146c2
…ations to TableDescriptor for existing tables (apache#3700) Signed-off-by: Duo Zhang <zhangduo@apache.org> Reviewed-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

HBASE-26454 CreateTableProcedure still relies on temp dir and renames… (
062ee1d
apache#3845) Signed-off-by: Duo Zhang <zhangduo@apache.org>

HBASE-26286: Add support for specifying store file tracker when resto…
899c700
…ring or cloning snapshot Closesapache#3851 Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Josh Elser <elserj@apache.org>

HBASE-26265 Update ref guide to mention the new store file tracker im… (
fed9993
apache#3942)

HBASE-26585 Add SFT configuration to META table descriptor when creat…
44718fa
…ing META (apache#3998) Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Josh Elser <elserj@apache.org> (cherry picked from commit baeb51f)

HBASE-26639 The implementation of TestMergesSplitsAddToTracker is pro…
e386198
…blematic (apache#4010) Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org> Conflicts: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/storefiletracker/TestStoreFileTracker.java

HBASE-26654 ModifyTableDescriptorProcedure shoud load TableDescriptor…
3631145
… while executing (apache#4034) Signed-off-by: GeorryHuang <huangzhuoyue@apache.org>

HBASE-26674 Should modify filesCompacting under storeWriteLock (apach…
84c833d
…e#4040) Signed-off-by: Josh Elser <elserj@apache.org>

HBASE-26675 Data race on Compactor.writer (apache#4035)
277e1c8
Signed-off-by: Xin Sun <ddupgs@gmail.com>

HBASE-26700 The way we bypass broken track file is not enough in Stor…
31219ba
…eFileListFile (apache#4055) Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

HBASE-26690 Modify FSTableDescriptors to not rely on renaming when wr…
922a516
…iting TableDescriptor (apache#4054) Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

HBASE-26587 Introduce a new Admin API to change SFT implementation (a…
068ab82
…pache#4030) (apache#4080) Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org> Reviewed-by: Josh Elser <elserj@apache.org>

HBASE-26673 Implement a shell command for change SFT implementation (a…
1d3d35c
…pache#4113) Signed-off-by: Duo Zhang <zhangduo@apache.org>

HBASE-26640 Reimplement master local region initialization to better …
a3d1419
…work with SFT (apache#4111) Signed-off-by: Josh Elser <elserj@apache.org> Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

HBASE-26707: Reduce number of renames during bulkload (apache#4066) (a…
44d4999
…pache#4122) Signed-off-by: Wellington Ramos Chevreuil <wchevreuil@apache.org> Conflicts: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBulkloadBase.java

HBASE-26611 Changing SFT implementation on disabled table is dangerous (
43cb672
apache#4082) Signed-off-by: Xiaolin Ha <haxiaolin@apache.org>

HBASE-26837 Set SFT config when creating TableDescriptor in TestClone…
5acfe6d
…SnapshotProcedure (apache#4226) Signed-off-by: Xiaolin Ha <haxiaolin@apache.org> (cherry picked from commit 5b28d76)

HBASE-26881 Backport HBASE-25368 to branch-2 (apache#4267)
541d748
Signed-off-by: Andrew Purtell <apurtell@apache.org>

apurtell force-pushed the HBASE-26826-2.5 branch 2 times, most recently from 4a7219c to f9fa857Compare March 26, 2022 16:20

apurtell force-pushed the HBASE-26826-2.5 branch from f9fa857 to 4e80f0eCompare March 26, 2022 16:30

apurtell merged commit 6902cb2 into apache:branch-2.5Mar 26, 2022

apurtell deleted the HBASE-26826-2.5 branch March 26, 2022 16:31

Uh oh!

Conversation

apurtell commented Mar 18, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

apurtell commented Mar 18, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Apache-HBase commented Mar 18, 2022

Uh oh!

Apache-HBase commented Mar 18, 2022

Uh oh!

Apache-HBase commented Mar 18, 2022

Uh oh!

apurtell commented Mar 18, 2022

Uh oh!

apurtell commented Mar 19, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

apurtell commented Mar 19, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Apache-HBase commented Mar 19, 2022

Uh oh!

Apache-HBase commented Mar 19, 2022

Uh oh!

Apache-HBase commented Mar 19, 2022

Uh oh!

wchevreuil commented Mar 22, 2022

Uh oh!

wchevreuil commented Mar 22, 2022

Uh oh!

joshelser commented Mar 23, 2022

Uh oh!

wchevreuil left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

apurtell commented Mar 25, 2022

Uh oh!

wchevreuil commented Mar 25, 2022

Uh oh!

apurtell commented Mar 25, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

apurtell commented Mar 25, 2022

Uh oh!

Apache-HBase commented Mar 25, 2022

Uh oh!

wchevreuil commented Mar 25, 2022

Uh oh!

Apache-HBase commented Mar 26, 2022

Uh oh!

Apache-HBase commented Mar 26, 2022

Uh oh!

Apache-HBase commented Mar 26, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

apurtell commented Mar 18, 2022•
edited
Loading

apurtell commented Mar 18, 2022•
edited
Loading

apurtell commented Mar 19, 2022•
edited
Loading

apurtell commented Mar 19, 2022•
edited
Loading

wchevreuil left a comment •
edited
Loading

apurtell commented Mar 25, 2022•
edited
Loading