HBASE-28385 make Scan estimates more realistic

[rmdmattingly]

Background

https://issues.apache.org/jira/browse/HBASE-28385

Let's say you're running a table scan with a throttle of 100MB/sec per RegionServer. Ideally your scans are going to pull down large results, often containing hundreds or thousands of blocks.

You will estimate each scan as costing a single block of read capacity, and if your quota is already exhausted then the server will evaluate the backoff required for your estimated consumption (1 block) to be available. This will often be ~1ms, causing your retries to basically be immediate.

Obviously it will routinely take much longer than 1ms for 100MB of IO to become available in the given configuration, so your retries will be destined to fail. At worst this can cause a saturation of your server's RPC layer, and at best this causes erroneous exhaustion of the client's retries.

Proposal

This makes two major changes:

1. It introduces a more complex estimation of scan workloads. Specifically:
   • We begin tracking the maxBlockBytesScanned, prevBlockBytesScanned, and prevBlockBytesScannedDiff for each scanner in the RegionScannerHolder.
   • Keep in mind that there is already a server configured maxResultSize for scanners, and a call sequence number which increments with each Scanner#next call, beginning with 0
   • With all of these inputs, we can differentiate between a new, growing, shrinking, or flat scan workload. We can also tell how much work it has done per RPC recently. With all of this information we're in a position to make an educated estimate about how much work the scan will do in its next call
2. It introduces a maximum scan estimate based on the quota's limit. This ensures that we never indefinitely block a scan based on its estimate against an otherwise idle quota.

Testing

I've deployed this to a test cluster and confirmed that large scan estimates quickly produced useful estimates and, consequently, meaningful waitInterval backoffs. I can also try to write a unit test which confirms this behavior.

cc @hgromer@eab148@bozzkar

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 36s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 0s	master passed
+1 💚	compile	2m 27s	master passed
+1 💚	checkstyle	0m 38s	master passed
+1 💚	spotless	0m 44s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 33s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 45s	the patch passed
-1 ❌	compile	2m 23s	hbase-server in the patch failed.
-0 ⚠️	javac	2m 23s	hbase-server in the patch failed.
-0 ⚠️	checkstyle	0m 36s	hbase-server: The patch generated 1 new + 8 unchanged - 0 fixed = 9 total (was 8)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	4m 55s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	0m 43s	patch has no errors when running spotless:check.
-1 ❌	spotbugs	1m 43s	hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
		_ Other Tests _
+1 💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		28m 23s

Reason	Tests
FindBugs	module:hbase-server
	org.apache.hadoop.hbase.regionserver.RSRpcServices$RegionScannerHolder.getMaxBlockBytesScanned() is unsynchronized, org.apache.hadoop.hbase.regionserver.RSRpcServices$RegionScannerHolder.setMaxBlockBytesScanned(long) is synchronized At RSRpcServices.java:synchronized At RSRpcServices.java:[line 457]

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 25b7de5aabd8 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / c4a02f7
Default Java	Eclipse Adoptium-11.0.17+8
compile	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt
checkstyle	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
spotbugs	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/1/artifact/yetus-general-check/output/new-spotbugs-hbase-server.html
Max. process+thread count	78 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 54s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 49s	master passed
+1 💚	compile	0m 50s	master passed
+1 💚	shadedjars	6m 1s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 31s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 27s	the patch passed
+1 💚	compile	1m 3s	the patch passed
+1 💚	javac	1m 3s	the patch passed
+1 💚	shadedjars	6m 33s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 35s	the patch passed
		_ Other Tests _
+1 💚	unit	291m 10s	hbase-server in the patch passed.
		320m 5s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 413eed8289f1 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / c4a02f7
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/1/testReport/
Max. process+thread count	4640 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 35s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 8s	master passed
+1 💚	compile	1m 3s	master passed
+1 💚	shadedjars	6m 59s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 31s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 6s	the patch passed
+1 💚	compile	0m 56s	the patch passed
+1 💚	javac	0m 56s	the patch passed
+1 💚	shadedjars	6m 4s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 32s	the patch passed
		_ Other Tests _
-1 ❌	unit	296m 36s	hbase-server in the patch failed.
		324m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 42dbbe60012e 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / c4a02f7
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/1/testReport/
Max. process+thread count	4875 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 42s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 44s	master passed
+1 💚	compile	3m 27s	master passed
+1 💚	checkstyle	0m 45s	master passed
+1 💚	spotless	0m 56s	branch has no errors when running spotless:check.
+1 💚	spotbugs	2m 30s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 22s	the patch passed
+1 💚	compile	3m 42s	the patch passed
+1 💚	javac	3m 42s	the patch passed
+1 💚	checkstyle	0m 53s	hbase-server: The patch generated 0 new + 9 unchanged - 1 fixed = 9 total (was 10)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	7m 0s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	1m 4s	patch has no errors when running spotless:check.
+1 💚	spotbugs	2m 40s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		41m 21s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 85a3472a467a 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / a1f09d1
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	78 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 13s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 29s	master passed
+1 💚	compile	1m 7s	master passed
+1 💚	shadedjars	6m 1s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 43s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 55s	the patch passed
+1 💚	compile	1m 3s	the patch passed
+1 💚	javac	1m 3s	the patch passed
+1 💚	shadedjars	6m 1s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 45s	the patch passed
		_ Other Tests _
+1 💚	unit	333m 7s	hbase-server in the patch passed.
		362m 4s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 4877f01bb292 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / a1f09d1
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/2/testReport/
Max. process+thread count	4544 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 10s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 7s	master passed
+1 💚	compile	0m 51s	master passed
+1 💚	shadedjars	5m 59s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 36s	the patch passed
+1 💚	compile	0m 57s	the patch passed
+1 💚	javac	0m 57s	the patch passed
+1 💚	shadedjars	5m 58s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	the patch passed
		_ Other Tests _
-1 ❌	unit	349m 55s	hbase-server in the patch failed.
		377m 9s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux e5899ff038b0 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / a1f09d1
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/2/testReport/
Max. process+thread count	4786 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[rmdmattingly]

I believe this test failure is unrelated

[rmdmattingly]

We've removed the minimum wait interval logic. I'm going to open a separate Jira regarding tiny backoffs and how we can improve them.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	patch	0m 3s	#5713 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#5713
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/3/console
versions	git=2.25.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	patch	0m 3s	#5713 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#5713
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/3/console
versions	git=2.25.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	patch	0m 3s	#5713 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#5713
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/3/console
versions	git=2.25.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	patch	0m 3s	#5713 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#5713
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/3/console
versions	git=2.17.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[rmdmattingly]

🤔 that's new. Just pushed a rebase

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 30s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 18s	master passed
+1 💚	compile	2m 28s	master passed
+1 💚	checkstyle	0m 38s	master passed
+1 💚	spotless	0m 45s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 32s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 49s	the patch passed
+1 💚	compile	2m 35s	the patch passed
+1 💚	javac	2m 35s	the patch passed
+1 💚	checkstyle	0m 35s	hbase-server: The patch generated 0 new + 9 unchanged - 1 fixed = 9 total (was 10)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	4m 59s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	0m 42s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 39s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 13s	The patch does not generate ASF License warnings.
		28m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux d87f279a58d5 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 936d267
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/4/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[bbeaudreault]

I feel like we should only double if the scan reached the previous limit.

[rmdmattingly]

Would you mind elaborating a bit? I don't know exactly what you mean

[bbeaudreault]

So every time a next() comes in, we double the previous BBS as the new limit. So let's say someone is doing a long scan with a caching of 1. So each time it's doing only 1 block of IO, but we are still doubling the new estimate. So it becomes harder to get a quota if we are always estimating 2x more than really fetching. The doubling is good for reaching a plateau quickly, but then it should stop.

For the 1 block/1 caching case, it might only be 64k vs 128kb estimate. Not a huge deal. But lets say we have a can that's doing 25mb per next(). To each time set the estimate to 50mb, that's quite a difference. Ideally once the scan itself plateaus in BBS, that should be the limit going forward. So let's say for the 25mb example, it does 64, 129, 256, etc up to 25. The next request the estimate is 50, but the resulting bbs is only 25. Ideally after that we just use 25 as the estimate going forward

[rmdmattingly]

So every time a next() comes in, we double the previous BBS as the new limit

Is this your proposal for what we should do, or your read on what the current estimation logic will do? If we're worried about overestimation, then I think this logic is actually worse than what you've described because we don't double the previous BBS — rather we multiply the max BBS by the next call sequence. So for 1 block/1 caching case, we'd quickly estimate at a, likely much higher, Math.min(maxScanerResultSize, maxScanEstimate). This is more of a feature than a bug though, because I think we'd prefer to overestimate against a saturated quota. For example, if we only manage to scan a few kb per next call because we're riding against the quota limit, but really the scan would prefer to pull down 100MB at a time, then we'll continue to use tiny estimations without the aforementioned estimate progression.

So, rather than risk falling into a cycle of quota saturation -> long running scans with tiny estimates -> tons of RPC calls -> quota saturation ..., I think we'd rather make the assumption describe in the comment here:

So we should estimate with the assumption that long-running scans
are appropriately configured to approach their maxScannerResultSize per RPC call

at the expense of potentially overestimating some scans. We also mitigate the risk of overestimation with the introduction of maxScanEstimate

[bbeaudreault]

I was describing what I thought the current implementation does. It seems like every new scan will increase the estimate u til we hit the server configured max. But I'd like to stop increasing when we stop exceeding the old limit. Imagine the following 4 next calls:

Estimate = 64k; Scanned = 64k
Estimate = 128k; Scanned = 128k
Estimate = 256k; Scanned = 128k
Estimate = 384k; Scanned = 128k

So the actually scanned amount had leveled out, but the estimate keeps increasing. Ideally it'd be like this instead:

Estimate = 64k; Scanned = 64k
Estimate = 128k; Scanned = 128k
Estimate = 256k; Scanned = 128k
Estimate = 128k; Scanned = 128k <- we didn't hit our last estimate, so stop over estimating.

Or something else that doesn't keep increasing. Of course the impact here is not huge yet, but for larger values it seems to matter a bit more. Unless I'm missing something

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 15s	master passed
+1 💚	compile	0m 52s	master passed
+1 💚	shadedjars	5m 32s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 50s	the patch passed
+1 💚	compile	0m 53s	the patch passed
+1 💚	javac	0m 53s	the patch passed
+1 💚	shadedjars	5m 29s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	the patch passed
		_ Other Tests _
+1 💚	unit	204m 42s	hbase-server in the patch passed.
		229m 37s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux aa8b4286d31a 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 936d267
Default Java	Eclipse Adoptium-17.0.10+7
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/4/testReport/
Max. process+thread count	4886 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/4/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 9s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 30s	master passed
+1 💚	compile	0m 42s	master passed
+1 💚	shadedjars	5m 9s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 29s	the patch passed
+1 💚	compile	0m 44s	the patch passed
+1 💚	javac	0m 44s	the patch passed
+1 💚	shadedjars	5m 9s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	the patch passed
		_ Other Tests _
-1 ❌	unit	231m 40s	hbase-server in the patch failed.
		253m 55s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 79f8f2536d7a 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 936d267
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/4/testReport/
Max. process+thread count	5343 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/4/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 34s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 17s	master passed
+1 💚	compile	0m 50s	master passed
+1 💚	shadedjars	5m 22s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 30s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 53s	the patch passed
+1 💚	compile	0m 53s	the patch passed
+1 💚	javac	0m 53s	the patch passed
+1 💚	shadedjars	5m 28s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	the patch passed
		_ Other Tests _
+1 💚	unit	231m 52s	hbase-server in the patch passed.
		256m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a74021ffa304 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 936d267
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/4/testReport/
Max. process+thread count	5071 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/4/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 40s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 11s	master passed
+1 💚	compile	3m 1s	master passed
+1 💚	checkstyle	0m 49s	master passed
+1 💚	spotless	1m 1s	branch has no errors when running spotless:check.
+1 💚	spotbugs	2m 12s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 13s	the patch passed
+1 💚	compile	3m 18s	the patch passed
+1 💚	javac	3m 18s	the patch passed
+1 💚	checkstyle	0m 46s	hbase-server: The patch generated 0 new + 9 unchanged - 1 fixed = 9 total (was 10)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	7m 23s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	1m 14s	patch has no errors when running spotless:check.
+1 💚	spotbugs	2m 22s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 11s	The patch does not generate ASF License warnings.
		40m 53s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux b43a44935c98 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 76c632c
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/5/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 41s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 28s	master passed
+1 💚	compile	1m 26s	master passed
+1 💚	shadedjars	7m 52s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 41s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 1s	the patch passed
+1 💚	compile	1m 16s	the patch passed
+1 💚	javac	1m 16s	the patch passed
+1 💚	shadedjars	8m 31s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 42s	the patch passed
		_ Other Tests _
+1 💚	unit	246m 48s	hbase-server in the patch passed.
		281m 0s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/5/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 08c376b73359 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 76c632c
Default Java	Eclipse Adoptium-17.0.10+7
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/5/testReport/
Max. process+thread count	5661 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/5/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 13s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 25s	master passed
+1 💚	compile	1m 5s	master passed
+1 💚	shadedjars	6m 2s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 44s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 59s	the patch passed
+1 💚	compile	1m 2s	the patch passed
+1 💚	javac	1m 2s	the patch passed
+1 💚	shadedjars	6m 3s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 44s	the patch passed
		_ Other Tests _
+1 💚	unit	332m 21s	hbase-server in the patch passed.
		361m 28s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux ac7918aee325 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 76c632c
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/5/testReport/
Max. process+thread count	4413 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/5/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 12s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 3s	master passed
+1 💚	compile	0m 47s	master passed
+1 💚	shadedjars	6m 5s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 28s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 31s	the patch passed
+1 💚	compile	1m 0s	the patch passed
+1 💚	javac	1m 0s	the patch passed
+1 💚	shadedjars	6m 2s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 29s	the patch passed
		_ Other Tests _
-1 ❌	unit	348m 54s	hbase-server in the patch failed.
		376m 16s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 9df6392b886f 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 76c632c
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/5/testReport/
Max. process+thread count	4382 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/5/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 12s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 32s	master passed
+1 💚	compile	0m 43s	master passed
+1 💚	shadedjars	5m 5s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 28s	the patch passed
+1 💚	compile	0m 42s	the patch passed
+1 💚	javac	0m 42s	the patch passed
+1 💚	shadedjars	5m 7s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	the patch passed
		_ Other Tests _
-1 ❌	unit	11m 6s	hbase-server in the patch failed.
		30m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 49c34d87e2fe 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 34b738d
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/6/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/6/testReport/
Max. process+thread count	1567 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/6/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 54s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 3s	master passed
+1 💚	compile	2m 30s	master passed
+1 💚	checkstyle	0m 36s	master passed
+1 💚	spotless	0m 53s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 52s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 50s	the patch passed
+1 💚	compile	3m 17s	the patch passed
+1 💚	javac	3m 17s	the patch passed
+1 💚	checkstyle	0m 43s	hbase-server: The patch generated 0 new + 9 unchanged - 1 fixed = 9 total (was 10)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	5m 48s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	0m 45s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 50s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 11s	The patch does not generate ASF License warnings.
		34m 1s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/6/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux adde281d546a 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 34b738d
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	81 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/6/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 35s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 53s	master passed
+1 💚	compile	0m 50s	master passed
+1 💚	shadedjars	5m 16s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 52s	the patch passed
+1 💚	compile	0m 51s	the patch passed
+1 💚	javac	0m 51s	the patch passed
+1 💚	shadedjars	5m 19s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	the patch passed
		_ Other Tests _
-1 ❌	unit	14m 5s	hbase-server in the patch failed.
		35m 25s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 6fa0c821ee0d 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 34b738d
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/6/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/6/testReport/
Max. process+thread count	1766 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/6/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 28s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 29s	master passed
+1 💚	compile	1m 6s	master passed
+1 💚	shadedjars	6m 7s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 36s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 12s	the patch passed
+1 💚	compile	0m 58s	the patch passed
+1 💚	javac	0m 58s	the patch passed
+1 💚	shadedjars	6m 19s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	the patch passed
		_ Other Tests _
-1 ❌	unit	12m 4s	hbase-server in the patch failed.
		36m 32s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/6/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 3cd7a9a3860f 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 34b738d
Default Java	Eclipse Adoptium-17.0.10+7
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/6/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/6/testReport/
Max. process+thread count	1796 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/6/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 12s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 26s	master passed
+1 💚	compile	0m 42s	master passed
+1 💚	shadedjars	5m 4s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 30s	the patch passed
+1 💚	compile	0m 42s	the patch passed
+1 💚	javac	0m 42s	the patch passed
+1 💚	shadedjars	5m 8s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	the patch passed
		_ Other Tests _
-1 ❌	unit	11m 9s	hbase-server in the patch failed.
		30m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux d74af2398ebd 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 34b738d
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/7/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/7/testReport/
Max. process+thread count	1565 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/7/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 34s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 38s	master passed
+1 💚	compile	2m 53s	master passed
+1 💚	checkstyle	0m 43s	master passed
+1 💚	spotless	0m 50s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 45s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 44s	the patch passed
+1 💚	compile	2m 59s	the patch passed
+1 💚	javac	2m 59s	the patch passed
+1 💚	checkstyle	0m 42s	hbase-server: The patch generated 0 new + 9 unchanged - 1 fixed = 9 total (was 10)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	5m 46s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	0m 49s	patch has no errors when running spotless:check.
+1 💚	spotbugs	2m 0s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 11s	The patch does not generate ASF License warnings.
		34m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/7/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 5f34e8757965 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 34b738d
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	80 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/7/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 25s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 5s	master passed
+1 💚	compile	0m 52s	master passed
+1 💚	shadedjars	5m 27s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 51s	the patch passed
+1 💚	compile	0m 53s	the patch passed
+1 💚	javac	0m 53s	the patch passed
+1 💚	shadedjars	5m 26s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	the patch passed
		_ Other Tests _
+1 💚	unit	203m 26s	hbase-server in the patch passed.
		227m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/7/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux cde69b120c06 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 34b738d
Default Java	Eclipse Adoptium-17.0.10+7
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/7/testReport/
Max. process+thread count	5088 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/7/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 34s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 49s	master passed
+1 💚	compile	0m 51s	master passed
+1 💚	shadedjars	5m 15s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 58s	the patch passed
+1 💚	compile	0m 54s	the patch passed
+1 💚	javac	0m 54s	the patch passed
+1 💚	shadedjars	5m 18s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	the patch passed
		_ Other Tests _
+1 💚	unit	232m 2s	hbase-server in the patch passed.
		256m 6s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 467273a6f0b3 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / 34b738d
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/7/testReport/
Max. process+thread count	5259 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/7/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[rmdmattingly]

I believe the two test failures on the latest build are unrelated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 37s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 57s	master passed
+1 💚	compile	2m 28s	master passed
+1 💚	checkstyle	0m 34s	master passed
+1 💚	spotless	0m 42s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 33s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 46s	the patch passed
+1 💚	compile	2m 29s	the patch passed
+1 💚	javac	2m 29s	the patch passed
+1 💚	checkstyle	0m 37s	hbase-server: The patch generated 0 new + 9 unchanged - 1 fixed = 9 total (was 10)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	4m 48s	Patch does not cause any errors with Hadoop 3.3.6.
+1 💚	spotless	0m 43s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 39s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 10s	The patch does not generate ASF License warnings.
		28m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/8/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 50f02278a732 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / beafd33
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	77 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/8/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 25s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 50s	master passed
+1 💚	compile	0m 52s	master passed
+1 💚	shadedjars	5m 28s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 47s	the patch passed
+1 💚	compile	0m 51s	the patch passed
+1 💚	javac	0m 51s	the patch passed
+1 💚	shadedjars	5m 29s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	the patch passed
		_ Other Tests _
+1 💚	unit	204m 26s	hbase-server in the patch passed.
		228m 18s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/8/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux bbf01b6c7a25 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / beafd33
Default Java	Eclipse Adoptium-17.0.10+7
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/8/testReport/
Max. process+thread count	5324 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/8/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 43s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 52s	master passed
+1 💚	compile	0m 38s	master passed
+1 💚	shadedjars	5m 38s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 26s	the patch passed
+1 💚	compile	0m 39s	the patch passed
+1 💚	javac	0m 39s	the patch passed
+1 💚	shadedjars	5m 36s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	the patch passed
		_ Other Tests _
+1 💚	unit	223m 47s	hbase-server in the patch passed.
		247m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/8/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux d9f3fe995387 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / beafd33
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/8/testReport/
Max. process+thread count	5683 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/8/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 35s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 10s	master passed
+1 💚	compile	0m 51s	master passed
+1 💚	shadedjars	5m 20s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 28s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 55s	the patch passed
+1 💚	compile	0m 51s	the patch passed
+1 💚	javac	0m 51s	the patch passed
+1 💚	shadedjars	5m 17s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	the patch passed
		_ Other Tests _
+1 💚	unit	232m 3s	hbase-server in the patch passed.
		256m 39s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/8/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5713
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 5d75f4d268c7 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / beafd33
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/8/testReport/
Max. process+thread count	4941 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5713/8/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.