HBASE-26263 [Rolling Upgrading] Persist the StoreFileTracker configur… by GeorryHuang · Pull Request #3700 · apache/hbase

GeorryHuang · 2021-09-27T02:56:48Z

…ations to TableDescriptor for existing tables

Apache9 · 2021-09-27T03:31:12Z

OK, we have the same logic when migrating the rs group config...

So let me see if we could have a more robust way, not your problem...

Apache-HBase · 2021-09-27T03:45:33Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 27s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ HBASE-26067 Compile Tests _
+1 💚	mvninstall	4m 10s	HBASE-26067 passed
+1 💚	compile	3m 13s	HBASE-26067 passed
+1 💚	checkstyle	1m 7s	HBASE-26067 passed
+1 💚	spotbugs	2m 10s	HBASE-26067 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 37s	the patch passed
+1 💚	compile	3m 13s	the patch passed
+1 💚	javac	3m 13s	the patch passed
-0 ⚠️	checkstyle	1m 4s	hbase-server: The patch generated 1 new + 89 unchanged - 0 fixed = 90 total (was 89)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	18m 27s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	2m 15s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 16s	The patch does not generate ASF License warnings.
		47m 50s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3700
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux af42b066821c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-26067 / `ab03ee1`
Default Java	AdoptOpenJDK-1.8.0_282-b08
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	96 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2021-09-27T03:47:09Z

After thinking, I think here we'd better introduce a special procedure to do this, as store file tracker is more critical than rs group. The procedure will be similiar to ModifyTableProcedure, but it does not need to reopen all the regions.

Apache-HBase · 2021-09-27T05:56:48Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 39s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ HBASE-26067 Compile Tests _
+1 💚	mvninstall	4m 54s	HBASE-26067 passed
+1 💚	compile	1m 20s	HBASE-26067 passed
+1 💚	shadedjars	8m 48s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 44s	HBASE-26067 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 21s	the patch passed
+1 💚	compile	1m 18s	the patch passed
+1 💚	javac	1m 18s	the patch passed
+1 💚	shadedjars	8m 39s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 41s	the patch passed
		_ Other Tests _
-1 ❌	unit	144m 32s	hbase-server in the patch failed.
		179m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3700
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 298dfbc61365 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-26067 / `ab03ee1`
Default Java	AdoptOpenJDK-11.0.10+9
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/1/testReport/
Max. process+thread count	3794 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

GeorryHuang · 2021-09-27T06:12:36Z

After thinking, I think here we'd better introduce a special procedure to do this, as store file tracker is more critical than rs group. The procedure will be similiar to ModifyTableProcedure, but it does not need to reopen all the regions.

I will refacor this background thread later. Let me check whether the logic of SFT changes in other places also need to use procedure.

Apache-HBase · 2021-09-27T07:10:26Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 13s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ HBASE-26067 Compile Tests _
+1 💚	mvninstall	4m 34s	HBASE-26067 passed
+1 💚	compile	1m 4s	HBASE-26067 passed
+1 💚	shadedjars	9m 5s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 39s	HBASE-26067 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 5s	the patch passed
+1 💚	compile	1m 2s	the patch passed
+1 💚	javac	1m 2s	the patch passed
+1 💚	shadedjars	9m 7s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	the patch passed
		_ Other Tests _
+1 💚	unit	219m 24s	hbase-server in the patch passed.
		252m 47s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3700
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 263bae8d43ea 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-26067 / `ab03ee1`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/1/testReport/
Max. process+thread count	3612 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2021-09-27T10:26:33Z

After thinking, I think here we'd better introduce a special procedure to do this, as store file tracker is more critical than rs group. The procedure will be similiar to ModifyTableProcedure, but it does not need to reopen all the regions.
I will refacor this background thread later. Let me check whether the logic of SFT changes in other places also need to use procedure.

The typical way to change SFT is through ModifyTableProcedure...

wchevreuil

Is this only relevant for upgrades between different versions of SFTs? Because I guess it wouldn't make much difference for upgrades between SFT and "non-SFT" supported versions.

wchevreuil · 2021-09-27T21:12:23Z

+ }
+ }, "StoreFileTrackerChecking");
+ STFChecking.setDaemon(true);
+ STFChecking.start();


This may cause too many threads to be submitted at once during initialisation, on clusters with many tables. And could we have some sort of "rolling-upgrade" to be set that would trigger this code, instead of doing this within every master initialization? I guess even if we do refactor this to be run as procedures, it would be worth avoid unnecessary runs.

Agree with you. Let me see if there is any way to solve it.

GeorryHuang · 2021-09-28T09:23:25Z

Is this only relevant for upgrades between different versions of SFTs? Because I guess it wouldn't make much difference for upgrades between SFT and "non-SFT" supported versions.

Yes, It relevant for upgrading different SFTs implementations.

GeorryHuang · 2021-09-28T09:35:53Z

@Apache9 Thanks for your suggestion, I think we should introduce a SilentModifyTableProcedure to use the procedure framework to update tableDescriptor instead of updating tableDescriptor directly to avoid critical race and lead to data loss. I will upload the code for you to review later

Apache-HBase · 2021-09-28T10:18:17Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 37s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ HBASE-26067 Compile Tests _
+1 💚	mvninstall	3m 57s	HBASE-26067 passed
+1 💚	compile	3m 12s	HBASE-26067 passed
+1 💚	checkstyle	1m 5s	HBASE-26067 passed
+1 💚	spotbugs	2m 3s	HBASE-26067 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 44s	the patch passed
+1 💚	compile	3m 11s	the patch passed
+1 💚	javac	3m 11s	the patch passed
-0 ⚠️	checkstyle	1m 5s	hbase-server: The patch generated 1 new + 89 unchanged - 0 fixed = 90 total (was 89)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	18m 11s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	2m 15s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		48m 20s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3700
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux a460eb703995 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-26067 / `ab03ee1`
Default Java	AdoptOpenJDK-1.8.0_282-b08
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	96 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/2/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-09-28T12:24:21Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 27s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ HBASE-26067 Compile Tests _
+1 💚	mvninstall	4m 23s	HBASE-26067 passed
+1 💚	compile	1m 12s	HBASE-26067 passed
+1 💚	shadedjars	8m 16s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	HBASE-26067 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 15s	the patch passed
+1 💚	compile	1m 11s	the patch passed
+1 💚	javac	1m 11s	the patch passed
+1 💚	shadedjars	8m 15s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	the patch passed
		_ Other Tests _
+1 💚	unit	143m 7s	hbase-server in the patch passed.
		174m 24s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3700
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 39f648cadf6b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-26067 / `ab03ee1`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/2/testReport/
Max. process+thread count	3823 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-09-28T13:46:35Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 4s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ HBASE-26067 Compile Tests _
+1 💚	mvninstall	4m 30s	HBASE-26067 passed
+1 💚	compile	1m 3s	HBASE-26067 passed
+1 💚	shadedjars	9m 7s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 38s	HBASE-26067 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 6s	the patch passed
+1 💚	compile	1m 3s	the patch passed
+1 💚	javac	1m 3s	the patch passed
+1 💚	shadedjars	9m 9s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	the patch passed
		_ Other Tests _
-1 ❌	unit	223m 36s	hbase-server in the patch failed.
		256m 39s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3700
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 2e0c1609696a 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-26067 / `ab03ee1`
Default Java	AdoptOpenJDK-1.8.0_282-b08
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/2/testReport/
Max. process+thread count	3382 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

GeorryHuang · 2021-10-11T08:19:05Z

Please take a look. UTs will be pushed later

Apache-HBase · 2021-10-11T09:11:14Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 34s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ HBASE-26067 Compile Tests _
+1 💚	mvninstall	4m 13s	HBASE-26067 passed
+1 💚	compile	3m 31s	HBASE-26067 passed
+1 💚	checkstyle	1m 6s	HBASE-26067 passed
+1 💚	spotbugs	2m 13s	HBASE-26067 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 56s	the patch passed
+1 💚	compile	3m 43s	the patch passed
+1 💚	javac	3m 43s	the patch passed
-0 ⚠️	checkstyle	1m 9s	hbase-server: The patch generated 12 new + 95 unchanged - 0 fixed = 107 total (was 95)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	21m 45s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	2m 27s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 13s	The patch does not generate ASF License warnings.
		53m 34s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3700
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 95338f48b02b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-26067 / `c4d7d28`
Default Java	AdoptOpenJDK-1.8.0_282-b08
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	96 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/4/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-11T11:13:19Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 26s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ HBASE-26067 Compile Tests _
+1 💚	mvninstall	4m 33s	HBASE-26067 passed
+1 💚	compile	1m 13s	HBASE-26067 passed
+1 💚	shadedjars	8m 28s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 44s	HBASE-26067 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 26s	the patch passed
+1 💚	compile	1m 16s	the patch passed
+1 💚	javac	1m 16s	the patch passed
+1 💚	shadedjars	8m 14s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	the patch passed
		_ Other Tests _
-1 ❌	unit	143m 41s	hbase-server in the patch failed.
		175m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3700
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 4dd59c0aa01f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-26067 / `c4d7d28`
Default Java	AdoptOpenJDK-11.0.10+9
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/4/testReport/
Max. process+thread count	4138 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/4/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-10-11T12:33:41Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 1s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ HBASE-26067 Compile Tests _
+1 💚	mvninstall	4m 20s	HBASE-26067 passed
+1 💚	compile	1m 4s	HBASE-26067 passed
+1 💚	shadedjars	9m 6s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 39s	HBASE-26067 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 8s	the patch passed
+1 💚	compile	1m 3s	the patch passed
+1 💚	javac	1m 3s	the patch passed
+1 💚	shadedjars	9m 2s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	the patch passed
		_ Other Tests _
-1 ❌	unit	223m 26s	hbase-server in the patch failed.
		256m 7s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3700
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux b81c064415e4 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-26067 / `c4d7d28`
Default Java	AdoptOpenJDK-1.8.0_282-b08
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/4/testReport/
Max. process+thread count	2939 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/4/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9

In general I like the approach to introduce a chore for upgrading and also a new procedure for updating the procedure.

But the problem here is the inheritance hierarchy of the procedures. We have a lot of states for ModifyTableProcedure, which are not needed for the upgrading procedure, and also, it will be safer if we do the check on whether we still need to update the descriptor in the procedure.

So let me 'steal' part of your work here, to reimplement the migrating of rs group on master branch, and then do a rebase so you could make use of the code to implement this PR cleaner.

WDYT?

Thanks.

GeorryHuang · 2021-10-11T23:39:08Z

In general I like the approach to introduce a chore for upgrading and also a new procedure for updating the procedure.
But the problem here is the inheritance hierarchy of the procedures. We have a lot of states for ModifyTableProcedure, which are not needed for the upgrading procedure, and also, it will be safer if we do the check on whether we still need to update the descriptor in the procedure.
So let me 'steal' part of your work here, to reimplement the migrating of rs group on master branch, and then do a rebase so you could make use of the code to implement this PR cleaner.
WDYT?
Thanks.

I agree with you.

…racking logic Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

) Signed-off-by: Duo Zhang <zhangduo@apache.org>

…g from different store file tracker implementations (apache#3656) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

…iptor when creating table (apache#3666) Signed-off-by: Duo Zhang <zhangduo@apache.org>

… file tracker implementation (apache#3665) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

Apache-HBase · 2021-11-05T03:56:32Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 26s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ HBASE-26067 Compile Tests _
+1 💚	mvninstall	3m 46s	HBASE-26067 passed
+1 💚	compile	3m 15s	HBASE-26067 passed
+1 💚	checkstyle	1m 5s	HBASE-26067 passed
+1 💚	spotbugs	2m 2s	HBASE-26067 passed
-0 ⚠️	patch	2m 11s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 46s	the patch passed
+1 💚	compile	3m 8s	the patch passed
+1 💚	javac	3m 8s	the patch passed
+1 💚	checkstyle	1m 5s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	19m 14s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	2m 15s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 14s	The patch does not generate ASF License warnings.
		48m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/14/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3700
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 84d216c98471 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-26067 / `3240b4b`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	95 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/14/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-11-05T06:04:04Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 27s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ HBASE-26067 Compile Tests _
+1 💚	mvninstall	4m 24s	HBASE-26067 passed
+1 💚	compile	1m 13s	HBASE-26067 passed
+1 💚	shadedjars	8m 6s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 41s	HBASE-26067 passed
-0 ⚠️	patch	9m 1s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 25s	the patch passed
+1 💚	compile	1m 14s	the patch passed
+1 💚	javac	1m 14s	the patch passed
+1 💚	shadedjars	8m 19s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 41s	the patch passed
		_ Other Tests _
-1 ❌	unit	144m 6s	hbase-server in the patch failed.
		175m 48s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/14/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3700
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux b54ff7def614 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-26067 / `3240b4b`
Default Java	AdoptOpenJDK-11.0.10+9
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/14/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/14/testReport/
Max. process+thread count	3950 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/14/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-11-05T07:22:21Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 2s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ HBASE-26067 Compile Tests _
+1 💚	mvninstall	4m 16s	HBASE-26067 passed
+1 💚	compile	1m 4s	HBASE-26067 passed
+1 💚	shadedjars	9m 5s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	HBASE-26067 passed
-0 ⚠️	patch	9m 53s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 20s	the patch passed
+1 💚	compile	1m 6s	the patch passed
+1 💚	javac	1m 6s	the patch passed
+1 💚	shadedjars	9m 1s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	the patch passed
		_ Other Tests _
-1 ❌	unit	221m 12s	hbase-server in the patch failed.
		254m 4s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/14/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3700
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux ca77529d8484 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-26067 / `3240b4b`
Default Java	AdoptOpenJDK-1.8.0_282-b08
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/14/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/14/testReport/
Max. process+thread count	3615 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/14/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

GeorryHuang · 2021-11-05T09:33:37Z

Now checkstyle has been fixed, but UT failed. It seems to be related to replication, it should not be caused by my commit.

Apache-HBase · 2021-11-05T16:20:18Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 26s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ HBASE-26067 Compile Tests _
+1 💚	mvninstall	3m 48s	HBASE-26067 passed
+1 💚	compile	3m 11s	HBASE-26067 passed
+1 💚	checkstyle	1m 4s	HBASE-26067 passed
+1 💚	spotbugs	2m 4s	HBASE-26067 passed
-0 ⚠️	patch	2m 14s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 50s	the patch passed
+1 💚	compile	3m 12s	the patch passed
+1 💚	javac	3m 12s	the patch passed
+1 💚	checkstyle	1m 6s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	19m 18s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	2m 15s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 14s	The patch does not generate ASF License warnings.
		48m 35s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/15/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3700
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 1320d9db8428 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-26067 / `3240b4b`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	95 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/15/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-11-05T18:26:01Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 27s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ HBASE-26067 Compile Tests _
+1 💚	mvninstall	4m 21s	HBASE-26067 passed
+1 💚	compile	1m 12s	HBASE-26067 passed
+1 💚	shadedjars	8m 14s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 41s	HBASE-26067 passed
-0 ⚠️	patch	9m 9s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 29s	the patch passed
+1 💚	compile	1m 14s	the patch passed
+1 💚	javac	1m 14s	the patch passed
+1 💚	shadedjars	8m 10s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 41s	the patch passed
		_ Other Tests _
+1 💚	unit	142m 42s	hbase-server in the patch passed.
		174m 20s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/15/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3700
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux c351c451edaf 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-26067 / `3240b4b`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/15/testReport/
Max. process+thread count	3893 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/15/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-11-05T19:46:16Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 4s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ HBASE-26067 Compile Tests _
+1 💚	mvninstall	4m 23s	HBASE-26067 passed
+1 💚	compile	1m 5s	HBASE-26067 passed
+1 💚	shadedjars	9m 9s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 38s	HBASE-26067 passed
-0 ⚠️	patch	9m 57s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 18s	the patch passed
+1 💚	compile	1m 4s	the patch passed
+1 💚	javac	1m 4s	the patch passed
+1 💚	shadedjars	9m 6s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	the patch passed
		_ Other Tests _
-1 ❌	unit	221m 11s	hbase-server in the patch failed.
		254m 24s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/15/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3700
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 5279a6ddf163 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	HBASE-26067 / `3240b4b`
Default Java	AdoptOpenJDK-1.8.0_282-b08
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/15/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/15/testReport/
Max. process+thread count	3464 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3700/15/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2021-11-06T14:15:54Z

The failed UTs seems not related. Let me merge.

…ations to TableDescriptor for existing tables (#3700) Signed-off-by: Duo Zhang <zhangduo@apache.org> Reviewed-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

…ations to TableDescriptor for existing tables (apache#3700) Signed-off-by: Duo Zhang <zhangduo@apache.org> Reviewed-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

…others) to branch-2.5 Previous cherry picks: commit 6aaef89 HBASE-26064 Introduce a StoreFileTracker to abstract the store file tracking logic commit 43b40e9 HBASE-25988 Store the store file list by a file apache#3578) commit 6e05376 HBASE-26079 Use StoreFileTracker when splitting and merging apache#3617) commit 090b2fe HBASE-26224 HBASE-26224 Introduce a MigrationStoreFileTracker to support migratin… apache#3656) commit 0ee1689 HBASE-26246 Persist the StoreFileTracker configurations to TableDescriptor when creating table apache#3666) commit 2052e80 HBASE-26248 Should find a suitable way to let users specify the store… apache#3665) commit 5ff0f98 HBASE-26264 Add more checks to prevent misconfiguration on store file… apache#3681) commit fc4f6d1 HBASE-26280 HBASE-26280 Use store file tracker when snapshoting apache#3685) commit 06db852 HBASE-26326 CreateTableProcedure fails when FileBasedStoreFileTracker… apache#3721) commit e4e7cf8 HBASE-26386 Refactor StoreFileTracker implementations to expose the s… apache#3774) commit 08d1171 HBASE-26328 Clone snapshot doesn't load reference files into FILE SFT impl apache#3749) commit 8bec26e HBASE-26263 [Rolling Upgrading] Persist the StoreFileTracker configur… apache#3700) commit a288365 HBASE-26271: Cleanup the broken store files under data directory apache#3786) commit d00b5fa HBASE-26454 CreateTableProcedure still relies on temp dir and renames… apache#3845) commit 771e552 HBASE-26286: Add support for specifying store file tracker when restoring or cloning snapshot commit f16b7b1 HBASE-26265 Update ref guide to mention the new store file tracker im… apache#3942) commit 755b3b4 HBASE-26585 Add SFT configuration to META table descriptor when creating META apache#3998) commit 39c42c7 HBASE-26639 The implementation of TestMergesSplitsAddToTracker is pro… apache#4010) commit 6e1f5b7 HBASE-26586 Should not rely on the global config when setting SFT implementation for a table while upgrading apache#4006) commit f1dd865 HBASE-26654 ModifyTableDescriptorProcedure shoud load TableDescriptor… apache#4034) commit 8fbc9a2 HBASE-26674 Should modify filesCompacting under storeWriteLock apache#4040) commit 5aa0fd2 HBASE-26675 Data race on Compactor.writer apache#4035) commit 3021c58 HBASE-26700 The way we bypass broken track file is not enough in Stor… apache#4055) commit a8b68c9 HBASE-26690 Modify FSTableDescriptors to not rely on renaming when wr… apache#4054) commit dffeb8e HBASE-26587 Introduce a new Admin API to change SFT implementation (#… apache#4080) commit b265fe5 HBASE-26673 Implement a shell command for change SFT implementation apache#4113) commit 4cdb380 HBASE-26640 Reimplement master local region initialization to better … apache#4111) commit 77bb153 HBASE-26707: Reduce number of renames during bulkload (apache#4066) apache#4122) commit a4b192e HBASE-26611 Changing SFT implementation on disabled table is dangerous apache#4082) commit d3629bb HBASE-26837 Set SFT config when creating TableDescriptor in TestClone… apache#4226) commit 541d748 HBASE-26881 Backport HBASE-25368 to branch-2 (apache#4267) Fixups for precommit error prone, checkstyle, and javadoc warnings after applying cherry picks. Signed-off-by: Josh Elser <elserj@apache.org> Reviewed-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

…ations to TableDescriptor for existing tables (#3700) Signed-off-by: Duo Zhang <zhangduo@apache.org> Reviewed-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

…others) to branch-2.5 Previous cherry picks: commit 6aaef89 HBASE-26064 Introduce a StoreFileTracker to abstract the store file tracking logic commit 43b40e9HBASE-25988 Store the store file list by a file #3578) commit 6e05376HBASE-26079 Use StoreFileTracker when splitting and merging #3617) commit 090b2feHBASE-26224 HBASE-26224 Introduce a MigrationStoreFileTracker to support migratin… #3656) commit 0ee1689HBASE-26246 Persist the StoreFileTracker configurations to TableDescriptor when creating table #3666) commit 2052e80HBASE-26248 Should find a suitable way to let users specify the store… #3665) commit 5ff0f98HBASE-26264 Add more checks to prevent misconfiguration on store file… #3681) commit fc4f6d1HBASE-26280 HBASE-26280 Use store file tracker when snapshoting #3685) commit 06db852HBASE-26326 CreateTableProcedure fails when FileBasedStoreFileTracker… #3721) commit e4e7cf8HBASE-26386 Refactor StoreFileTracker implementations to expose the s… #3774) commit 08d1171HBASE-26328 Clone snapshot doesn't load reference files into FILE SFT impl #3749) commit 8bec26eHBASE-26263 [Rolling Upgrading] Persist the StoreFileTracker configur… #3700) commit a288365HBASE-26271: Cleanup the broken store files under data directory #3786) commit d00b5faHBASE-26454 CreateTableProcedure still relies on temp dir and renames… #3845) commit 771e552HBASE-26286: Add support for specifying store file tracker when restoring or cloning snapshot commit f16b7b1HBASE-26265 Update ref guide to mention the new store file tracker im… #3942) commit 755b3b4HBASE-26585 Add SFT configuration to META table descriptor when creating META #3998) commit 39c42c7HBASE-26639 The implementation of TestMergesSplitsAddToTracker is pro… #4010) commit 6e1f5b7HBASE-26586 Should not rely on the global config when setting SFT implementation for a table while upgrading #4006) commit f1dd865HBASE-26654 ModifyTableDescriptorProcedure shoud load TableDescriptor… #4034) commit 8fbc9a2HBASE-26674 Should modify filesCompacting under storeWriteLock #4040) commit 5aa0fd2HBASE-26675 Data race on Compactor.writer #4035) commit 3021c58HBASE-26700 The way we bypass broken track file is not enough in Stor… #4055) commit a8b68c9HBASE-26690 Modify FSTableDescriptors to not rely on renaming when wr… #4054) commit dffeb8eHBASE-26587 Introduce a new Admin API to change SFT implementation (#… #4080) commit b265fe5HBASE-26673 Implement a shell command for change SFT implementation #4113) commit 4cdb380HBASE-26640 Reimplement master local region initialization to better … #4111) commit 77bb153HBASE-26707: Reduce number of renames during bulkload (#4066) #4122) commit a4b192e HBASE-26611 Changing SFT implementation on disabled table is dangerous #4082) commit d3629bbHBASE-26837 Set SFT config when creating TableDescriptor in TestClone… #4226) commit 541d748HBASE-26881 Backport HBASE-25368 to branch-2 (#4267) Fixups for precommit error prone, checkstyle, and javadoc warnings after applying cherry picks. Signed-off-by: Josh Elser <elserj@apache.org> Reviewed-by: Wellington Ramos Chevreuil <wchevreuil@apache.org>

…ations to TableDescriptor for existing tables (apache#3700) Signed-off-by: Duo Zhang <zhangduo@apache.org> Reviewed-by: Wellington Ramos Chevreuil <wchevreuil@apache.org> Change-Id: I32214acc8704fd0a76c8f6c9001238890c1a8b3a

Apache9 reviewed Sep 27, 2021
View reviewed changes

Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java Outdated

wchevreuil reviewed Sep 27, 2021
View reviewed changes

GeorryHuang closed this Sep 28, 2021

GeorryHuang reopened this Sep 28, 2021

Apache9 force-pushed the HBASE-26067 branch from ab03ee1 to c4d7d28Compare October 5, 2021 14:32

GeorryHuang force-pushed the HBASE-26263 branch from 2addcdc to 9604a61Compare October 11, 2021 08:16

Apache9 reviewed Oct 11, 2021
View reviewed changes

Apache9and others added 6 commits October 14, 2021 23:11

HBASE-26064 Introduce a StoreFileTracker to abstract the store file t…
d75b0ce
…racking logic Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

HBASE-25988 Store the store file list by a file (apache#3578)
cab6140
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

HBASE-26079 Use StoreFileTracker when splitting and merging (apache#3617
708b7c1
) Signed-off-by: Duo Zhang <zhangduo@apache.org>

HBASE-26224 Introduce a MigrationStoreFileTracker to support migratin…
41ed7e2
…g from different store file tracker implementations (apache#3656) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

HBASE-26246 Persist the StoreFileTracker configurations to TableDescr…
f2addb1
…iptor when creating table (apache#3666) Signed-off-by: Duo Zhang <zhangduo@apache.org>

HBASE-26248 Should find a suitable way to let users specify the store…
5c35744
… file tracker implementation (apache#3665) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

Fixed Checkstyle
f84ea4a

Apache9 merged commit 49a9211 into apache:HBASE-26067Nov 6, 2021

Apache9 mentioned this pull request Nov 6, 2021
HBASE-26271: Cleanup the broken store files under data directory #3786
Merged

apurtell mentioned this pull request Mar 18, 2022
Backport StoreFileTracker (HBASE-26067, HBASE-26584, and others) to branch-2.5 #4241
Merged

Uh oh!

Conversation

GeorryHuang commented Sep 27, 2021

Uh oh!

Uh oh!

Apache9 commented Sep 27, 2021

Uh oh!

Apache-HBase commented Sep 27, 2021

Uh oh!

Apache9 commented Sep 27, 2021

Uh oh!

Apache-HBase commented Sep 27, 2021

Uh oh!

GeorryHuang commented Sep 27, 2021

Uh oh!

Apache-HBase commented Sep 27, 2021

Uh oh!

Apache9 commented Sep 27, 2021

Uh oh!

wchevreuil left a comment

Choose a reason for hiding this comment

Uh oh!

wchevreuilSep 27, 2021

Choose a reason for hiding this comment

Uh oh!

GeorryHuangSep 28, 2021

Choose a reason for hiding this comment

Uh oh!

GeorryHuang commented Sep 28, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

GeorryHuang commented Sep 28, 2021

Uh oh!

Apache-HBase commented Sep 28, 2021

Uh oh!

Apache-HBase commented Sep 28, 2021

Uh oh!

Apache-HBase commented Sep 28, 2021

Uh oh!

GeorryHuang commented Oct 11, 2021

Uh oh!

Apache-HBase commented Oct 11, 2021

Uh oh!

Apache-HBase commented Oct 11, 2021

Uh oh!

Apache-HBase commented Oct 11, 2021

Uh oh!

Apache9 left a comment

Choose a reason for hiding this comment

Uh oh!

GeorryHuang commented Oct 11, 2021

Uh oh!

Apache-HBase commented Nov 5, 2021

Uh oh!

Apache-HBase commented Nov 5, 2021

Uh oh!

Apache-HBase commented Nov 5, 2021

Uh oh!

GeorryHuang commented Nov 5, 2021

Uh oh!

Apache-HBase commented Nov 5, 2021

Uh oh!

Apache-HBase commented Nov 5, 2021

Uh oh!

Apache-HBase commented Nov 5, 2021

Uh oh!

Apache9 commented Nov 6, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

GeorryHuang commented Sep 28, 2021•
edited
Loading