Skip to content

HBASE-23304: RPCs needed for client meta information lookup (apache#904) - #1098

Merged
apurtell merged 4 commits into
apache:branch-2from
bharathv:HBASE-23304-branch-2
Jan 29, 2020
Merged

HBASE-23304: RPCs needed for client meta information lookup (apache#904)#1098
apurtell merged 4 commits into
apache:branch-2from
bharathv:HBASE-23304-branch-2

Conversation

@bharathv

Copy link
Copy Markdown
Contributor

This patch implements the RPCs needed for the meta information
lookup during connection init. New tests added to cover the RPC
code paths. HBASE-23305 builds on this to implement the client
side logic.

Fixed a bunch of checkstyle nits around the places the patch
touches.

Signed-off-by: Andrew Purtell apurtell@apache.org

This patch implements a simple cache that all the masters
can lookup to serve cluster ID to clients. Active HMaster
is still responsible for creating it but all the masters
will read it from fs to serve clients.
RPCs exposing it will come in a separate patch as a part of
HBASE-18095.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Guangxu Cheng <guangxucheng@gmail.com>
(cherry picked from commit c2e01f2)
…ache#812)
Currently we just track whether an active master exists.
It helps to also track the address of the active master in
all the masters to help serve the client RPC requests to
know which master is active.
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit efebb84)
* HBASE-23281: Track meta region changes on masters
This patch adds a simple cache that tracks the meta region replica
locations. It keeps an eye on the region movements so that the
cached locations are not stale.
This information is used for servicing client RPCs for connections
that use master based registry (HBASE-18095). The RPC end points
will be added in a separate patch.
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
(cherry picked from commit 8571d38)
* HBASE-23304: RPCs needed for client meta information lookup
This patch implements the RPCs needed for the meta information
lookup during connection init. New tests added to cover the RPC
code paths. HBASE-23305 builds on this to implement the client
side logic.
Fixed a bunch of checkstyle nits around the places the patch
touches.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
(cherry picked from commit 4f8fbba)
@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 35sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 1sNo case conflicting files found.
+0 🆗prototool0m 0sprototool was not available.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 19 new or modified test files.
_ branch-2 Compile Tests _
+0 🆗mvndep0m 16sMaven dependency ordering for branch
+1 💚mvninstall5m 22sbranch-2 passed
+1 💚compile2m 25sbranch-2 passed
+1 💚checkstyle2m 15sbranch-2 passed
+1 💚shadedjars4m 16sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc1m 34sbranch-2 passed
+0 🆗spotbugs0m 34sUsed deprecated FindBugs config; considering switching to SpotBugs.
+1 💚findbugs7m 51sbranch-2 passed
_ Patch Compile Tests _
+0 🆗mvndep0m 15sMaven dependency ordering for patch
+1 💚mvninstall4m 55sthe patch passed
+1 💚compile2m 24sthe patch passed
+1 💚cc2m 24sthe patch passed
+1 💚javac2m 24sthe patch passed
+1 💚checkstyle0m 12sThe patch passed checkstyle in hbase-protocol-shaded
+1 💚checkstyle0m 37shbase-client: The patch generated 0 new + 194 unchanged - 1 fixed = 194 total (was 195)
+1 💚checkstyle0m 13sThe patch passed checkstyle in hbase-zookeeper
+1 💚checkstyle1m 12shbase-server: The patch generated 0 new + 119 unchanged - 13 fixed = 119 total (was 132)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚shadedjars4m 19spatch has no errors when building our shaded downstream artifacts.
+1 💚hadoopcheck15m 39sPatch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 💚hbaseprotoc2m 17sthe patch passed
+1 💚javadoc1m 33sthe patch passed
+1 💚findbugs8m 20sthe patch passed
_ Other Tests _
+1 💚unit0m 42shbase-protocol-shaded in the patch passed.
+1 💚unit3m 45shbase-client in the patch passed.
+1 💚unit0m 53shbase-zookeeper in the patch passed.
+1 💚unit153m 34shbase-server in the patch passed.
+1 💚asflicense2m 12sThe patch does not generate ASF License warnings.
235m 0s
SubsystemReport/Notes
DockerClient=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1098/1/artifact/out/Dockerfile
GITHUB PR#1098
JIRA IssueHBASE-23304
Optional Testsdupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool
unameLinux 1aaa31531bb2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build toolmaven
Personality/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1098/out/precommit/personality/provided.sh
git revisionbranch-2 / e0f9133
Default Java1.8.0_181
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1098/1/testReport/
Max. process+thread count6066 (vs. ulimit of 10000)
modulesC: hbase-protocol-shaded hbase-client hbase-zookeeper hbase-server U: .
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1098/1/console
versionsgit=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@wchevreuilwchevreuil left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ndimiduk

Copy link
Copy Markdown
Member

@bharathv@apurtell What exactly was this commit? The message looks like it contains everything from the feature branch, but the changed files is just HBASE-23304.

Anyway, we haven't merged the feature branch to master yet; we're not quite ready to back port to branch-2. I think this should be reverted.

ndimiduk added a commit to ndimiduk/hbase that referenced this pull request Feb 3, 2020
@bharathv
bharathv deleted the HBASE-23304-branch-2 branch February 3, 2020 22:47
@bharathv

Copy link
Copy Markdown
ContributorAuthor

Ya, so this is a squashed commit up until HBASE-23304. I'm ok with reverting it. I can submit a single PR with all the commits later, once the feature branch merges with master.

ndimiduk added a commit to ndimiduk/hbase that referenced this pull request Feb 3, 2020
@apurtell

Copy link
Copy Markdown
Contributor

Copying my comment from the JIRA:

There were multiple PRs opened for the same set of changes. I merged the one @bharathv indicated.

If you want to do it per JIRA per branch, fine. One PR per JIRA per branch. However that's a lot of noise for what it's worth.

I think it better one PR/merge for the whole backport per branch, with commit message incorporating all relevant JIRA references. These changes for the most part aren't useful in isolation.

@ndimiduk

Copy link
Copy Markdown
Member

I see your reasoning. Squishing all the commits makes accounting more difficult, IMHO. The larger issue is that we're still stabilizing the tests vs. master; none of these commits have landed on master yet.

@ndimiduk

Copy link
Copy Markdown
Member

I should say -- I propose one PR per branch containing all the commits, but land it using "rebase and merge" instead of "squash and merge" so that the individual commits are preserved. That make the accounting I refer to simpler. Agreed there's no need for one PR per commit.

ndimiduk added a commit that referenced this pull request Feb 5, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@bharathv@Apache-HBase@ndimiduk@apurtell@wchevreuil