HBASE-27687: support consumption of `block bytes scanned` in operation quota by rmdmattingly · Pull Request #5654 · apache/hbase

rmdmattingly · 2024-01-25T23:59:06Z

See https://issues.apache.org/jira/browse/HBASE-27687

We've found that result size is not a great indication of a request's workload for the server. For example, as is, one may set a user to have a 100mb/machine throttle and then find that the given user is able to generate far more than 100mb of IO against any given machine if their result sizes are significantly smaller than the IO required to facilitate the request.

This PR introduces hbase.quota.use.block.bytes.scanned, a configurable boolean (defaulted to false) which, if true, will cause quotas to consume read availability via block bytes scanned rather than result size. Extending the aforementioned example, this should help to ensure that a user throttled to 100mb of IO/machine will not be able to exceed 100mb of IO/machine.

I've deployed this to a test cluster and confirmed the intended behavior. For example, below is a chart of the MB/sec IO footprint across 4 RegionServers serving a heavily filtered scan workload coming from a user with a 100MB/sec throttle configured. As you can see, in the beginning, each host is far exceeding 100MB/sec of IO (because it is measuring throttle usage via result sizes, which are small post-filtering). In this timeframe we roll each host to use a build of HBase which includes this PR's changeset (and a configuration of hbase.quota.use.block.bytes.scanned to true), and consequently we see the erroneously large IO workloads all throttle back to basically exactly 100MB/sec:

@bbeaudreault @hgromer @eab148 @bozzkar

Apache-HBase · 2024-01-26T00:38:35Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 33s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 2s	master passed
+1 💚	compile	2m 28s	master passed
+1 💚	checkstyle	0m 37s	master passed
+1 💚	spotless	0m 43s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 31s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 43s	the patch passed
-1 ❌	compile	2m 23s	hbase-server in the patch failed.
-0 ⚠️	javac	2m 23s	hbase-server in the patch failed.
+1 💚	checkstyle	0m 36s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	4m 56s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	0m 42s	patch has no errors when running spotless:check.
-1 ❌	spotbugs	1m 39s	hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
		_ Other Tests _
+1 💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		28m 3s

Reason	Tests
FindBugs	module:hbase-server
	Nullcheck of rpcCall at line 3659 of value previously dereferenced in org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RpcController, ClientProtos$ScanRequest) At RSRpcServices.java:3659 of value previously dereferenced in org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RpcController, ClientProtos$ScanRequest) At RSRpcServices.java:[line 3659]

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux ad51ed540d7c 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `1c5b5ff`
Default Java	Eclipse Adoptium-11.0.17+8
compile	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt
spotbugs	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/1/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html
Max. process+thread count	83 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-01-26T04:13:15Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 26s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 1s	master passed
+1 💚	compile	0m 44s	master passed
+1 💚	shadedjars	5m 30s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 46s	the patch passed
+1 💚	compile	0m 43s	the patch passed
+1 💚	javac	0m 43s	the patch passed
+1 💚	shadedjars	5m 31s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	the patch passed
		_ Other Tests _
-1 ❌	unit	219m 1s	hbase-server in the patch failed.
		242m 31s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux e1a922232a2c 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `1c5b5ff`
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/1/testReport/
Max. process+thread count	4746 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-01-26T05:00:53Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 38s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 11s	master passed
+1 💚	compile	0m 48s	master passed
+1 💚	shadedjars	5m 41s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 30s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 0s	the patch passed
+1 💚	compile	0m 46s	the patch passed
+1 💚	javac	0m 46s	the patch passed
+1 💚	shadedjars	5m 37s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 29s	the patch passed
		_ Other Tests _
-1 ❌	unit	264m 52s	hbase-server in the patch failed.
		290m 20s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 13d90650f749 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `1c5b5ff`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/1/testReport/
Max. process+thread count	5243 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-01-26T13:38:30Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 34s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 52s	master passed
+1 💚	compile	2m 24s	master passed
+1 💚	checkstyle	0m 33s	master passed
+1 💚	spotless	0m 41s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 28s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 45s	the patch passed
+1 💚	compile	2m 24s	the patch passed
+1 💚	javac	2m 24s	the patch passed
+1 💚	checkstyle	0m 36s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	4m 46s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	0m 42s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 37s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 10s	The patch does not generate ASF License warnings.
		27m 51s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux d307e35421dc 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `9c79a7f`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	81 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-01-26T17:42:08Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 36s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 3s	master passed
+1 💚	compile	0m 48s	master passed
+1 💚	shadedjars	5m 5s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 43s	the patch passed
+1 💚	compile	0m 49s	the patch passed
+1 💚	javac	0m 49s	the patch passed
+1 💚	shadedjars	5m 3s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
-1 ❌	unit	247m 57s	hbase-server in the patch failed.
		271m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 993fa79e9000 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `9c79a7f`
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/2/testReport/
Max. process+thread count	4728 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-01-26T18:17:39Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 41s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 4s	master passed
+1 💚	compile	0m 53s	master passed
+1 💚	shadedjars	5m 56s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 9s	the patch passed
+1 💚	compile	0m 56s	the patch passed
+1 💚	javac	0m 56s	the patch passed
+1 💚	shadedjars	5m 55s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 28s	the patch passed
		_ Other Tests _
-1 ❌	unit	280m 34s	hbase-server in the patch failed.
		307m 1s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 8183d78f9c96 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `9c79a7f`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/2/testReport/
Max. process+thread count	4329 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

rmdmattingly · 2024-01-28T15:48:18Z

Build number 2 failed above sort of inexplicably AFAICT, so the unit test report is not available. I believe I've fixed the legitimate unit test failures from build number 1. I've force pushed to restart the build process.

Apache-HBase · 2024-01-28T16:18:22Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 12s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 4s	master passed
+1 💚	compile	2m 28s	master passed
+1 💚	checkstyle	0m 37s	master passed
+1 💚	spotless	0m 44s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 31s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 43s	the patch passed
+1 💚	compile	2m 27s	the patch passed
+1 💚	javac	2m 27s	the patch passed
+1 💚	checkstyle	0m 37s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	4m 48s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	0m 41s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 40s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		27m 52s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 1b24937e8331 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `3b2e981`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	82 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/3/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-01-28T20:06:57Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 13s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 31s	master passed
+1 💚	compile	0m 43s	master passed
+1 💚	shadedjars	5m 3s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 25s	the patch passed
+1 💚	compile	0m 41s	the patch passed
+1 💚	javac	0m 41s	the patch passed
+1 💚	shadedjars	4m 58s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
-1 ❌	unit	232m 43s	hbase-server in the patch failed.
		256m 49s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 188c6537bd62 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `3b2e981`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/3/testReport/
Max. process+thread count	5249 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-01-28T20:08:44Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 38s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 52s	master passed
+1 💚	compile	0m 49s	master passed
+1 💚	shadedjars	5m 4s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 47s	the patch passed
+1 💚	compile	0m 49s	the patch passed
+1 💚	javac	0m 49s	the patch passed
+1 💚	shadedjars	5m 7s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	the patch passed
		_ Other Tests _
+1 💚	unit	236m 0s	hbase-server in the patch passed.
		258m 34s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 16e2d74862b0 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `3b2e981`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/3/testReport/
Max. process+thread count	4738 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-01-28T20:40:58Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 44s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 1s	master passed
+1 💚	compile	2m 27s	master passed
+1 💚	checkstyle	0m 37s	master passed
+1 💚	spotless	0m 43s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 31s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 42s	the patch passed
+1 💚	compile	2m 25s	the patch passed
+1 💚	javac	2m 25s	the patch passed
+1 💚	checkstyle	0m 36s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	4m 47s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	0m 41s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 38s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		31m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 887fa50fef46 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `3b2e981`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	82 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/4/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-01-29T00:20:35Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 31s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 22s	master passed
+1 💚	compile	0m 39s	master passed
+1 💚	shadedjars	4m 59s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 27s	the patch passed
+1 💚	compile	0m 40s	the patch passed
+1 💚	javac	0m 40s	the patch passed
+1 💚	shadedjars	4m 58s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	the patch passed
		_ Other Tests _
-1 ❌	unit	229m 1s	hbase-server in the patch failed.
		250m 46s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 99f6f27f1305 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `3b2e981`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/4/testReport/
Max. process+thread count	5268 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/4/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-01-29T00:28:49Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 32s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 7s	master passed
+1 💚	compile	0m 49s	master passed
+1 💚	shadedjars	5m 4s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 44s	the patch passed
+1 💚	compile	0m 48s	the patch passed
+1 💚	javac	0m 48s	the patch passed
+1 💚	shadedjars	5m 3s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
+1 💚	unit	235m 40s	hbase-server in the patch passed.
		258m 57s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a03241a8caaf 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `3b2e981`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/4/testReport/
Max. process+thread count	4694 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/4/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

rmdmattingly · 2024-01-29T16:18:02Z

The test failure here appears to be unrelated and transient

bbeaudreault · 2024-02-02T18:07:55Z

 if (metricsRegionServer != null) {
 long blockBytesScanned =
 context != null ? context.getBlockBytesScanned() - blockBytesScannedBefore : 0;
+ quota.addBlockBytesScanned(blockBytesScanned);


this may not be a huge issue, but i think this shouldn't be within the metricsRegionServer != null check. Maybe move this and the above line up outside

Also you missed adding this to append

bbeaudreault · 2024-02-02T18:08:53Z

 long after = EnvironmentEdgeManager.currentTime();
 long blockBytesScanned =
 context != null ? context.getBlockBytesScanned() - blockBytesScannedBefore : 0;
+ quota.addBlockBytesScanned(blockBytesScanned);


bbeaudreault · 2024-02-02T19:45:09Z

 spaceQuotaEnforcement);
 }
 } finally {
+ if (context != null) {


I dont think this works out so well, because of how some of the methods are abstracted. For example, increment can be called from the single mutate method and also from doNonAtomicRegionMutation (which is called from multi). So for increments, we'd double account them.
I might recommend only adding calls to addBlockBytesScanned wherever there are existing calls to getBlockBytesScanned() for metrics. As I mentioned in another comment, you'll want to move the getBlockBytesScanned() outside the if (metrics != null) checks, but otherwise those are probably the best places to add the quota calls if possible.

Hmm actually maybe it's better to keep this here and to instead handle avoiding double counting append/increment, i.e. by passing in a isMulti boolean or something

bbeaudreault · 2024-02-02T22:27:04Z

- readConsumed = estimateConsume(OperationType.GET, numReads, 100);
- readConsumed += estimateConsume(OperationType.SCAN, numScans, 1000);
+ if (useBlockBytesScanned) {
+ writeConsumed = estimateConsume(OperationType.MUTATE, numWrites, 100);


this is a nit, but can you move the writeConsumed out of the if blocks so we don't duplicate?

Apache-HBase · 2024-02-02T23:04:11Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 21s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 58s	master passed
+1 💚	compile	2m 32s	master passed
+1 💚	checkstyle	0m 38s	master passed
+1 💚	spotless	0m 43s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 31s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 41s	the patch passed
+1 💚	compile	2m 24s	the patch passed
+1 💚	javac	2m 24s	the patch passed
+1 💚	checkstyle	0m 37s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	4m 49s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	0m 41s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 39s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		28m 38s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 4bbf90c56de8 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c4e332a`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	78 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/5/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-02-03T02:51:11Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 20s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 43s	master passed
+1 💚	compile	0m 49s	master passed
+1 💚	shadedjars	5m 4s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 46s	the patch passed
+1 💚	compile	0m 49s	the patch passed
+1 💚	javac	0m 49s	the patch passed
+1 💚	shadedjars	5m 4s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	the patch passed
		_ Other Tests _
+1 💚	unit	229m 54s	hbase-server in the patch passed.
		255m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 55c9fc37be44 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c4e332a`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/5/testReport/
Max. process+thread count	4671 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/5/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-02-03T02:55:07Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 9s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 33s	master passed
+1 💚	compile	0m 42s	master passed
+1 💚	shadedjars	5m 2s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 27s	the patch passed
+1 💚	compile	0m 41s	the patch passed
+1 💚	javac	0m 41s	the patch passed
+1 💚	shadedjars	4m 57s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	the patch passed
		_ Other Tests _
-1 ❌	unit	234m 53s	hbase-server in the patch failed.
		259m 39s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 70215ae2ce60 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c4e332a`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/5/testReport/
Max. process+thread count	5485 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/5/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-02-03T03:23:18Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 12s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 46s	master passed
+1 💚	compile	2m 25s	master passed
+1 💚	checkstyle	0m 35s	master passed
+1 💚	spotless	0m 41s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 30s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 44s	the patch passed
+1 💚	compile	2m 25s	the patch passed
+1 💚	javac	2m 25s	the patch passed
+1 💚	checkstyle	0m 34s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	4m 46s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	0m 41s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 37s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 10s	The patch does not generate ASF License warnings.
		26m 58s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/6/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 2a22aeb75b03 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c4e332a`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	81 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/6/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-02-03T07:08:15Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 36s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 41s	master passed
+1 💚	compile	0m 47s	master passed
+1 💚	shadedjars	5m 3s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 44s	the patch passed
+1 💚	compile	0m 48s	the patch passed
+1 💚	javac	0m 48s	the patch passed
+1 💚	shadedjars	5m 3s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
-1 ❌	unit	229m 18s	hbase-server in the patch failed.
		251m 56s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux c78f5e356d1f 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c4e332a`
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/6/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/6/testReport/
Max. process+thread count	4701 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/6/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-02-03T07:10:40Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 30s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 26s	master passed
+1 💚	compile	0m 42s	master passed
+1 💚	shadedjars	5m 0s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 26s	the patch passed
+1 💚	compile	0m 40s	the patch passed
+1 💚	javac	0m 40s	the patch passed
+1 💚	shadedjars	4m 59s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
-1 ❌	unit	232m 42s	hbase-server in the patch failed.
		254m 22s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a4321290416b 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c4e332a`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/6/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/6/testReport/
Max. process+thread count	5298 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/6/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-02-06T15:33:02Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 14s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 1s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 49s	master passed
+1 💚	compile	2m 26s	master passed
+1 💚	checkstyle	0m 37s	master passed
+1 💚	spotless	0m 42s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 30s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 47s	the patch passed
+1 💚	compile	2m 25s	the patch passed
+1 💚	javac	2m 25s	the patch passed
+1 💚	checkstyle	0m 37s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	4m 44s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	0m 42s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 37s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 10s	The patch does not generate ASF License warnings.
		27m 27s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/7/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux da3917a8ab3e 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `16de74c`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/7/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-02-06T19:21:25Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 34s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 50s	master passed
+1 💚	compile	0m 47s	master passed
+1 💚	shadedjars	5m 5s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 47s	the patch passed
+1 💚	compile	0m 49s	the patch passed
+1 💚	javac	0m 49s	the patch passed
+1 💚	shadedjars	5m 2s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	the patch passed
		_ Other Tests _
+1 💚	unit	232m 26s	hbase-server in the patch passed.
		255m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 0d9efcca3c06 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `16de74c`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/7/testReport/
Max. process+thread count	4747 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/7/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-02-06T20:01:14Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 34s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 23s	master passed
+1 💚	compile	0m 51s	master passed
+1 💚	shadedjars	6m 8s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 29s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 59s	the patch passed
+1 💚	compile	0m 44s	the patch passed
+1 💚	javac	0m 44s	the patch passed
+1 💚	shadedjars	5m 52s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
-1 ❌	unit	269m 41s	hbase-server in the patch failed.
		295m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5654
JIRA Issue	HBASE-27687
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux f819b55027b5 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:28:33 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `16de74c`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/7/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/7/testReport/
Max. process+thread count	5925 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5654/7/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

rmdmattingly · 2024-02-06T20:03:17Z

That test failure looks unrelated

…esponse size (#5654) Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>

…nned rather than response size (apache#5654) Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>

Consume block bytes scanned in operation quota
2c083b7

rmdmattingly commented Jan 26, 2024
View reviewed changes

Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/quotas/DefaultOperationQuota.java
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java

set minBlockSizeBytes in constructor
31a7882

fix warnings
e7c8a05

rmdmattingly force-pushed the HBASE-27687 branch from dd7adca to e7c8a05Compare January 28, 2024 15:47

test cleanup
433a699

bbeaudreault reviewed Feb 2, 2024
View reviewed changes

Simplify blockBytesScanned derivation. Use max of BBS and result size
7f660d4

bbeaudreault reviewed Feb 2, 2024
View reviewed changes

nit: remove duplicated write estimate logic
c36ab49

Use BBS as default. Makes tests a bit more defensive against flakiness
d167196

rmdmattingly force-pushed the HBASE-27687 branch from 9d07c44 to d167196Compare February 6, 2024 14:58

bbeaudreault approved these changes Feb 6, 2024
View reviewed changes

bbeaudreault merged commit afbf2ad into apache:masterFeb 6, 2024

bbeaudreault deleted the HBASE-27687 branch February 6, 2024 20:28

bbeaudreault pushed a commit that referenced this pull request Feb 6, 2024
HBASE-27687Enhance quotas to consume blockBytesScanned rather than r…
3ea4fef
…esponse size (#5654) Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>

bbeaudreault pushed a commit that referenced this pull request Feb 6, 2024
HBASE-27687Enhance quotas to consume blockBytesScanned rather than r…
876e782
…esponse size (#5654) Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>

bbeaudreault pushed a commit that referenced this pull request Feb 6, 2024
HBASE-27687Enhance quotas to consume blockBytesScanned rather than r…
22808cf
…esponse size (#5654) Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>

bbeaudreault pushed a commit to HubSpot/hbase that referenced this pull request Feb 6, 2024
HubSpot Backport: HBASE-27687 Enhance quotas to consume blockBytesSca…
a4159d3
…nned rather than response size (apache#5654) Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>

Uh oh!

Conversation

rmdmattingly commented Jan 25, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Apache-HBase commented Jan 26, 2024

Uh oh!

Apache-HBase commented Jan 26, 2024

Uh oh!

Apache-HBase commented Jan 26, 2024

Uh oh!

Apache-HBase commented Jan 26, 2024

Uh oh!

Apache-HBase commented Jan 26, 2024

Uh oh!

Apache-HBase commented Jan 26, 2024

Uh oh!

rmdmattingly commented Jan 28, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Apache-HBase commented Jan 28, 2024

Uh oh!

Apache-HBase commented Jan 28, 2024

Uh oh!

Apache-HBase commented Jan 28, 2024

Uh oh!

Apache-HBase commented Jan 28, 2024

Uh oh!

Apache-HBase commented Jan 29, 2024

Uh oh!

Apache-HBase commented Jan 29, 2024

Uh oh!

rmdmattingly commented Jan 29, 2024

Uh oh!

bbeaudreaultFeb 2, 2024

Choose a reason for hiding this comment

Uh oh!

bbeaudreaultFeb 2, 2024

Choose a reason for hiding this comment

Uh oh!

bbeaudreaultFeb 2, 2024

Choose a reason for hiding this comment

Uh oh!

bbeaudreaultFeb 2, 2024

Choose a reason for hiding this comment

Uh oh!

bbeaudreaultFeb 2, 2024

Choose a reason for hiding this comment

Uh oh!

bbeaudreaultFeb 2, 2024

Choose a reason for hiding this comment

Uh oh!

Apache-HBase commented Feb 2, 2024

Uh oh!

Apache-HBase commented Feb 3, 2024

Uh oh!

Apache-HBase commented Feb 3, 2024

Uh oh!

Apache-HBase commented Feb 3, 2024

Uh oh!

Apache-HBase commented Feb 3, 2024

Uh oh!

Apache-HBase commented Feb 3, 2024

Uh oh!

Apache-HBase commented Feb 6, 2024

Uh oh!

Apache-HBase commented Feb 6, 2024

Uh oh!

Apache-HBase commented Feb 6, 2024

Uh oh!

rmdmattingly commented Feb 6, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

rmdmattingly commented Jan 25, 2024•
edited
Loading

rmdmattingly commented Jan 28, 2024•
edited
Loading