HBASE-27523 Add BulkLoad bandwidth throttling

[SiCheng-Zheng]

No description provided.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 7s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 38s	master passed
+1 💚	compile	2m 26s	master passed
+1 💚	checkstyle	0m 37s	master passed
+1 💚	spotless	0m 43s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 29s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 27s	the patch passed
+1 💚	compile	2m 26s	the patch passed
+1 💚	javac	2m 26s	the patch passed
-0 ⚠️	checkstyle	0m 35s	hbase-server: The patch generated 5 new + 36 unchanged - 0 fixed = 41 total (was 36)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	8m 47s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
-1 ❌	spotless	0m 36s	patch has 68 errors when running spotless:check, run spotless:apply to fix.
+1 💚	spotbugs	1m 36s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		31m 21s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4975
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux f99e104b4986 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 7ed2cb9
Default Java	Eclipse Adoptium-11.0.17+8
checkstyle	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
spotless	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/1/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 35s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 38s	master passed
+1 💚	compile	0m 41s	master passed
+1 💚	shadedjars	3m 58s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 26s	the patch passed
+1 💚	compile	0m 42s	the patch passed
+1 💚	javac	0m 42s	the patch passed
+1 💚	shadedjars	3m 57s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 21s	the patch passed
		_ Other Tests _
-1 ❌	unit	200m 26s	hbase-server in the patch failed.
		219m 58s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4975
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 8f5ab45ca3c3 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 7ed2cb9
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/1/testReport/
Max. process+thread count	2530 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 42s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 30s	master passed
+1 💚	compile	0m 36s	master passed
+1 💚	shadedjars	4m 4s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 6s	the patch passed
+1 💚	compile	0m 36s	the patch passed
+1 💚	javac	0m 36s	the patch passed
+1 💚	shadedjars	4m 4s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	the patch passed
		_ Other Tests _
+1 💚	unit	208m 59s	hbase-server in the patch passed.
		228m 56s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4975
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux bccb61b9a0f3 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 7ed2cb9
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/1/testReport/
Max. process+thread count	2634 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	15m 36s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 22s	master passed
+1 💚	compile	2m 22s	master passed
+1 💚	checkstyle	0m 36s	master passed
+1 💚	spotless	0m 44s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 28s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 19s	the patch passed
+1 💚	compile	2m 23s	the patch passed
+1 💚	javac	2m 23s	the patch passed
-0 ⚠️	checkstyle	0m 35s	hbase-server: The patch generated 4 new + 36 unchanged - 0 fixed = 40 total (was 36)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 9s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
-1 ❌	spotless	0m 36s	patch has 69 errors when running spotless:check, run spotless:apply to fix.
-1 ❌	spotbugs	1m 37s	hbase-server generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
		_ Other Tests _
+1 💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		52m 3s

Reason	Tests
FindBugs	module:hbase-server
	Inconsistent synchronization of org.apache.hadoop.hbase.regionserver.throttle.BulkLoadThrottler.cycleStartTick; locked 75% of time Unsynchronized access at BulkLoadThrottler.java:75% of time Unsynchronized access at BulkLoadThrottler.java:[line 138]
	org.apache.hadoop.hbase.regionserver.throttle.BulkLoadThrottler.limitNextBytes(int) calls Thread.sleep() with a lock held At BulkLoadThrottler.java:lock held At BulkLoadThrottler.java:[line 104]

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4975
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 6ac6969bac47 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 8e72cc6
Default Java	Eclipse Adoptium-11.0.17+8
checkstyle	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
spotless	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/2/artifact/yetus-general-check/output/patch-spotless.txt
spotbugs	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/2/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	8m 22s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 32s	master passed
+1 💚	compile	0m 41s	master passed
+1 💚	shadedjars	4m 36s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 17s	the patch passed
+1 💚	compile	0m 42s	the patch passed
+1 💚	javac	0m 42s	the patch passed
+1 💚	shadedjars	4m 34s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	the patch passed
		_ Other Tests _
+1 💚	unit	196m 16s	hbase-server in the patch passed.
		227m 2s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4975
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux cd72950505dd 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 8e72cc6
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/2/testReport/
Max. process+thread count	2681 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 41s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 17s	master passed
+1 💚	compile	0m 34s	master passed
+1 💚	shadedjars	4m 35s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 48s	the patch passed
+1 💚	compile	0m 34s	the patch passed
+1 💚	javac	0m 34s	the patch passed
+1 💚	shadedjars	4m 35s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 21s	the patch passed
		_ Other Tests _
-1 ❌	unit	221m 25s	hbase-server in the patch failed.
		244m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4975
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 601ab3829d0d 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 8e72cc6
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/2/testReport/
Max. process+thread count	2650 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	docker	0m 1s	Docker failed to build yetus/hbase:e7d1a6a052.

Subsystem	Report/Notes
GITHUB PR	#4975
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/3/console
versions	git=2.17.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	docker	0m 0s	Docker failed to build yetus/hbase:e7d1a6a052.

Subsystem	Report/Notes
GITHUB PR	#4975
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/3/console
versions	git=2.17.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 58s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 40s	master passed
+1 💚	compile	2m 30s	master passed
+1 💚	checkstyle	0m 35s	master passed
+1 💚	spotless	0m 43s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 32s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 33s	the patch passed
+1 💚	compile	2m 36s	the patch passed
+1 💚	javac	2m 36s	the patch passed
+1 💚	checkstyle	0m 35s	the patch passed
+1 💚	whitespace	0m 1s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 40s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚	spotless	0m 42s	patch has no errors when running spotless:check.
-1 ❌	spotbugs	1m 45s	hbase-server generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
		_ Other Tests _
+1 💚	asflicense	0m 8s	The patch does not generate ASF License warnings.
		39m 39s

Reason	Tests
FindBugs	module:hbase-server
	Inconsistent synchronization of org.apache.hadoop.hbase.regionserver.throttle.BulkLoadThrottler.cycleStartTick; locked 75% of time Unsynchronized access at BulkLoadThrottler.java:75% of time Unsynchronized access at BulkLoadThrottler.java:[line 137]
	org.apache.hadoop.hbase.regionserver.throttle.BulkLoadThrottler.limitNextBytes(int) calls Thread.sleep() with a lock held At BulkLoadThrottler.java:lock held At BulkLoadThrottler.java:[line 104]

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4975
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 7eea93d3190e 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 55b4bbc
Default Java	Eclipse Adoptium-11.0.17+8
spotbugs	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/3/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4975/3/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[bbeaudreault]

Can we use existing ThroughputController interface for this?

[SiCheng-Zheng]

ThroughputController interface

Let me study the ThroughputController interface