Skip to content

HBASE-29005 Cannot split hbase:quota table when quota enforcement is enabled - #6501

Merged
ndimiduk merged 2 commits into
apache:masterfrom
ndimiduk:29005-master
Dec 5, 2024
Merged

HBASE-29005 Cannot split hbase:quota table when quota enforcement is enabled#6501
ndimiduk merged 2 commits into
apache:masterfrom
ndimiduk:29005-master

Conversation

@ndimiduk

Copy link
Copy Markdown
Member

When quotas are enabled, splitting the quota table fails due to the namespace region quotas system being partially initialized.

2024-11-28T14:11:38,672 ERROR [PEWorker-2 {}]
procedure2.ProcedureExecutor(1844): CODE-BUG: Uncaught runtime
exception: pid=7, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION,
hasLock=true; SplitTableRegionProcedure table=hbase:quota,
parent=3c1dbd551845ed2e8033b76570c6d8f6,
daughterA=a8f5e70728d93859d7c3d95aa5a0f63b,
daughterB=c71f49ad7e689a57742a9fe65b918d08
java.lang.NullPointerException: Cannot invoke
"java.util.concurrent.atomic.AtomicInteger.addAndGet(int)" because the
return value of "java.util.Map.get(Object)" is null
at org.apache.hadoop.hbase.namespace.NamespaceTableAndRegionInfo.incRegionCountForTable(NamespaceTableAndRegionInfo.java:101) ~[classes/:?]
at org.apache.hadoop.hbase.namespace.NamespaceStateManager.checkAndUpdateNamespaceRegionCount(NamespaceStateManager.java:95) ~[classes/:?]
at org.apache.hadoop.hbase.namespace.NamespaceAuditor.checkQuotaToSplitRegion(NamespaceAuditor.java:116) ~[classes/:?]
at org.apache.hadoop.hbase.quotas.MasterQuotaManager.onRegionSplit(MasterQuotaManager.java:544) ~[classes/:?]
at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.preSplitRegion(SplitTableRegionProcedure.java:607) ~[classes/:?]
at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) ~[classes/:?]
at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:98) ~[classes/:?]
at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[classes/:?]
at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:941) ~[classes/:?]
at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1825) ~[classes/:?]
at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1503)~[classes/:?]
at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:2154) ~[classes/:?]
at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[classes/:?]
at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:2181) ~[classes/:?]

See discussion on the dev list, https://lists.apache.org/thread/gdnjb1ys8m93lowykdrh4m7o55mgw3vo

@ndimiduk

Copy link
Copy Markdown
MemberAuthor

FYI @droudnitsky

*/
@Category({ MiscTests.class, LargeTests.class })
@RunWith(Parameterized.class)
public class TestSplitMergeQuotaTable {

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reduced scope of this test class to just cover the quota table. Previously It was for system tables and on branch-2 I was adding coverage for the namespace table. Turns out that we hard-code cannot split the namespace table, so that was futile.


@Parameterized.Parameters(name = "{1}")
public static Object[][] params() {
return new Object[][] { { Map.of(QuotaUtil.QUOTA_CONF_KEY, "false") },

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing both with and without the quotas feature enabled, so that we have confidence that someone who has temporarily disabled the feature is still able to operate on the table.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@ndimiduk

Copy link
Copy Markdown
MemberAuthor

[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.001 s - in org.apache.hadoop.hbase.TestSplitMergeQuotaTable

@ndimidukndimiduk changed the title HBASE-29005 Cannot split split system tables when quota enforcement is enabledHBASE-29005 Cannot split hbase:quota table when quota enforcement is enabledDec 4, 2024
@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 27sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ master Compile Tests _
+1 💚mvninstall3m 29smaster passed
+1 💚compile3m 13smaster passed
+1 💚checkstyle0m 41smaster passed
+1 💚spotbugs1m 46smaster passed
+1 💚spotless0m 52sbranch has no errors when running spotless:check.
-0 ⚠️patch0m 59sUsed diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 💚mvninstall3m 20sthe patch passed
+1 💚compile3m 20sthe patch passed
+1 💚javac3m 20sthe patch passed
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 38sthe patch passed
+1 💚spotbugs1m 50sthe patch passed
+1 💚hadoopcheck12m 7sPatch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚spotless0m 47spatch has no errors when running spotless:check.
_ Other Tests _
+1 💚asflicense0m 11sThe patch does not generate ASF License warnings.
39m 52s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6501/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#6501
Optional Testsdupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
unameLinux 61c0130e47ba 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 4a6a514
Default JavaEclipse Adoptium-17.0.11+9
Max. process+thread count84 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6501/4/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 28sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 28smaster passed
+1 💚compile1m 2smaster passed
+1 💚javadoc0m 31smaster passed
+1 💚shadedjars6m 5sbranch has no errors when building our shaded downstream artifacts.
-0 ⚠️patch6m 14sUsed diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 💚mvninstall3m 17sthe patch passed
+1 💚compile0m 59sthe patch passed
+1 💚javac0m 59sthe patch passed
+1 💚javadoc0m 30sthe patch passed
+1 💚shadedjars6m 1spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚unit221m 51shbase-server in the patch passed.
248m 4s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6501/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#6501
Optional Testsjavac javadoc unit compile shadedjars
unameLinux e06afcd0942b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 4a6a514
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6501/4/testReport/
Max. process+thread count4991 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6501/4/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@ndimiduk
ndimiduk merged commit 1880cb3 into apache:masterDec 5, 2024
@ndimiduk
ndimiduk deleted the 29005-master branch December 5, 2024 08:46
ndimiduk added a commit that referenced this pull request Dec 5, 2024
…enabled (#6501)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
ndimiduk added a commit that referenced this pull request Dec 5, 2024
…enabled (#6501)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
ndimiduk added a commit that referenced this pull request Dec 5, 2024
…enabled (#6501)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
ndimiduk added a commit that referenced this pull request Dec 5, 2024
…enabled (#6501)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
ndimiduk added a commit to HubSpot/hbase that referenced this pull request Dec 5, 2024
…enabled (apache#6501)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
ndimiduk added a commit to HubSpot/hbase that referenced this pull request Dec 9, 2024
…enabled (apache#6501)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
gvprathyusha6 pushed a commit to gvprathyusha6/hbase that referenced this pull request Dec 19, 2024
…enabled (apache#6501)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
mokai87 pushed a commit to mokai87/hbase that referenced this pull request Aug 7, 2025
…enabled (apache#6501)
Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@ndimiduk@Apache-HBase@bbeaudreault