Skip to content

HBASE-23055 Alter hbase:meta - #646

Closed
saintstack wants to merge 1 commit into
apache:branch-2from
saintstack:HBASE-23055
Closed

HBASE-23055 Alter hbase:meta#646
saintstack wants to merge 1 commit into
apache:branch-2from
saintstack:HBASE-23055

Conversation

@saintstack

Copy link
Copy Markdown
Contributor

Make it so hbase:meta can be altered. TableState for hbase:meta
is kept in Master. State is in-memory transient so if Master
fails, hbase:meta is ENABLED again. hbase:meta schema will be
bootstrapped from the filesystem. Changes to filesystem schema
are atomic so we should be ok if Master fails mid-edit (TBD)
Undoes a bunch of guards that prevented our being able to edit
hbase:meta. At minimmum, need to add in a bunch of WARNING.

TODO: Tests, more clarity around hbase:meta table state, and undoing
references to hard-coded hbase:meta regioninfo.

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
Throw illegal access exception if you try to use MetaTableAccessor
getting state of the hbase:meta table.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
TODO: getTableState needs work in Connection implemetnations. Presumes
state is in meta table for all tables. Uses MetaTableAccessor.
TODO: More cleanup in here and more cleanup in async versions.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
Change isTableDisabled/Enabled implementation to ask the Master instead.
This will give the Master's TableStateManager's opinion rather than
client figuring it for themselves reading meta table direct.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
TODO: Cleanup in here. Go to master for state, not to meta.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
Logging cleanup.

M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
Shutdown access.

M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
Just cleanup.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
Add state holder for hbase:meta.
Removed unused methods.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
Shut down access.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
Allow hbase:meta to be disabled.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
Allow hbase:meta to be enabled.

public TableState getTableState(TableName tableName) throws IOException {
// TODO: This doesn't work if tablename is hbase:meta. Need to ask Master.
// Other problems with this implementation are that it presumes state is
// available in Master. Would be good to hide how state is kept.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/hbase:meta/Master/

}
// TODO: This doesn't work if tablename is hbase:meta. Need to ask Master.
// Other problems with this implementation are that it presumes state is
// available in Master. Would be good to hide how state is kept.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Master/hbase:meta/

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
💙reexec1m 8sDocker mode activated.
_ Prechecks _
💚dupname0m 0sNo case conflicting files found.
💚hbaseanti0m 0sPatch does not have any anti-patterns.
💚@author0m 0sThe patch does not contain any @author tags.
💛test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-2 Compile Tests _
💙mvndep0m 13sMaven dependency ordering for branch
💚mvninstall5m 39sbranch-2 passed
💚compile2m 2sbranch-2 passed
💚checkstyle2m 50sbranch-2 passed
💚shadedjars4m 45sbranch has no errors when building our shaded downstream artifacts.
💚javadoc1m 33sbranch-2 passed
💙spotbugs0m 36sUsed deprecated FindBugs config; considering switching to SpotBugs.
💚findbugs6m 6sbranch-2 passed
_ Patch Compile Tests _
💙mvndep0m 14sMaven dependency ordering for patch
💚mvninstall5m 22sthe patch passed
💚compile2m 2sthe patch passed
💚javac2m 2sthe patch passed
💔checkstyle0m 27shbase-common: The patch generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10)
💔checkstyle0m 41shbase-client: The patch generated 6 new + 169 unchanged - 4 fixed = 175 total (was 173)
💔checkstyle1m 28shbase-server: The patch generated 2 new + 50 unchanged - 11 fixed = 52 total (was 61)
💚whitespace0m 0sThe patch has no whitespace issues.
💚shadedjars4m 42spatch has no errors when building our shaded downstream artifacts.
💚hadoopcheck16m 59sPatch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚javadoc1m 41sthe patch passed
💚findbugs6m 36sthe patch passed
_ Other Tests _
💚unit3m 6shbase-common in the patch passed.
💚unit2m 29shbase-client in the patch passed.
💚unit0m 52shbase-zookeeper in the patch passed.
💔unit235m 18shbase-server in the patch failed.
💚asflicense1m 33sThe patch does not generate ASF License warnings.
315m 54s
ReasonTests
Failed junit testshadoop.hbase.client.TestAdmin2
hadoop.hbase.client.TestAsyncTableAdminApi3
hadoop.hbase.TestSplitMerge
hadoop.hbase.client.TestAsyncTableAdminApi2
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/1/artifact/out/Dockerfile
GITHUB PR#646
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux 23dc5114e7cc 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-646/out/precommit/personality/provided.sh
git revisionbranch-2 / 0ed11fe
Default Java1.8.0_181
checkstylehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/1/artifact/out/diff-checkstyle-hbase-common.txt
checkstylehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/1/artifact/out/diff-checkstyle-hbase-client.txt
checkstylehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/1/artifact/out/diff-checkstyle-hbase-server.txt
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/1/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/1/testReport/
Max. process+thread count4523 (vs. ulimit of 10000)
modulesC: hbase-common hbase-client hbase-zookeeper hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/1/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

 Make it so hbase:meta can be altered. TableState for hbase:meta
is kept in Master. State is in-memory transient so if Master
fails, hbase:meta is ENABLED again. hbase:meta schema will be
bootstrapped from the filesystem. Changes to filesystem schema
are atomic so we should be ok if Master fails mid-edit (TBD)
Undoes a bunch of guards that prevented our being able to edit
hbase:meta. At minimmum, need to add in a bunch of WARNING.
TODO: Tests, more clarity around hbase:meta table state, and undoing
references to hard-coded hbase:meta regioninfo.
M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
Throw illegal access exception if you try to use MetaTableAccessor
getting state of the hbase:meta table.
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
For table state, go to master rather than go to meta direct. Going
to meta won't work for hbase;meta state. Puts load on Master.
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
Change isTableDisabled/Enabled implementation to ask the Master instead.
This will give the Master's TableStateManager's opinion rather than
client figuring it for themselves reading meta table direct.
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
TODO: Cleanup in here. Go to master for state, not to meta.
M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
Logging cleanup.
M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
Shutdown access.
M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
Just cleanup.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
Add state holder for hbase:meta.
Removed unused methods.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
Shut down access.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
Allow hbase:meta to be disabled.
M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
Allow hbase:meta to be enabled.
@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
💙reexec1m 35sDocker mode activated.
_ Prechecks _
💚dupname0m 0sNo case conflicting files found.
💚hbaseanti0m 0sPatch does not have any anti-patterns.
💚@author0m 0sThe patch does not contain any @author tags.
💚test4tests0m 0sThe patch appears to include 2 new or modified test files.
_ branch-2 Compile Tests _
💙mvndep0m 13sMaven dependency ordering for branch
💚mvninstall5m 44sbranch-2 passed
💚compile2m 0sbranch-2 passed
💚checkstyle2m 56sbranch-2 passed
💚shadedjars4m 38sbranch has no errors when building our shaded downstream artifacts.
💚javadoc1m 32sbranch-2 passed
💙spotbugs0m 34sUsed deprecated FindBugs config; considering switching to SpotBugs.
💚findbugs6m 4sbranch-2 passed
_ Patch Compile Tests _
💙mvndep0m 13sMaven dependency ordering for patch
💚mvninstall5m 15sthe patch passed
💚compile1m 59sthe patch passed
💚javac1m 59sthe patch passed
💔checkstyle0m 25shbase-common: The patch generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10)
💔checkstyle0m 43shbase-client: The patch generated 20 new + 349 unchanged - 5 fixed = 369 total (was 354)
💔checkstyle1m 35shbase-server: The patch generated 6 new + 372 unchanged - 14 fixed = 378 total (was 386)
💚whitespace0m 0sThe patch has no whitespace issues.
💚shadedjars4m 37spatch has no errors when building our shaded downstream artifacts.
💚hadoopcheck17m 20sPatch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚javadoc1m 30sthe patch passed
💔findbugs1m 24shbase-client generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
_ Other Tests _
💚unit2m 47shbase-common in the patch passed.
💚unit3m 34shbase-client in the patch passed.
💚unit0m 48shbase-zookeeper in the patch passed.
💔unit29m 35shbase-server in the patch failed.
💚asflicense0m 53sThe patch does not generate ASF License warnings.
109m 50s
ReasonTests
FindBugsmodule:hbase-client
Call to org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos$TableState.equals(org.apache.hadoop.hbase.client.TableState$State) in org.apache.hadoop.hbase.client.RawAsyncHBaseAdmin.lambda$null$32(TableState$State, MasterProtos$GetTableStateResponse) At RawAsyncHBaseAdmin.java:MasterProtos$GetTableStateResponse) At RawAsyncHBaseAdmin.java:[line 683]
Failed junit testshadoop.hbase.client.TestAsyncAccessControlAdminApi
SubsystemReport/Notes
DockerClient=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/artifact/out/Dockerfile
GITHUB PR#646
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
unameLinux a6021653900c 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-646/out/precommit/personality/provided.sh
git revisionbranch-2 / 6fe7063
Default Java1.8.0_181
checkstylehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/artifact/out/diff-checkstyle-hbase-common.txt
checkstylehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/artifact/out/diff-checkstyle-hbase-client.txt
checkstylehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/artifact/out/diff-checkstyle-hbase-server.txt
findbugshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/artifact/out/new-findbugs-hbase-client.html
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/artifact/out/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/testReport/
Max. process+thread count680 (vs. ulimit of 10000)
modulesC: hbase-common hbase-client hbase-zookeeper hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@saintstack@Apache-HBase