Uh oh!
There was an error while loading. Please reload this page.
HBASE-27280 Add mutual authentication support to TLS - #4724
Closed
bbeaudreault wants to merge 1 commit into
Closed
Conversation
Apache-HBase
commented
Aug 23, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Aug 23, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Aug 23, 2022
🎊 +1 overall
This message was automatically generated. |
| * revision</a> | ||
| */ | ||
| @InterfaceAudience.Private | ||
| final class HBaseHostnameVerifier implements HostnameVerifier { |
ContributorAuthor
There was a problem hiding this comment.
this file was copied verbatim
| * revision</a> | ||
| */ | ||
| @InterfaceAudience.Private | ||
| public class HBaseTrustManager extends X509ExtendedTrustManager { |
ContributorAuthor
There was a problem hiding this comment.
this file was copied verbatim
| * "https://github.com/apache/zookeeper/blob/5820d10d9dc58c8e12d2e25386fdf92acb360359/zookeeper-server/src/test/java/org/apache/zookeeper/common/ZKHostnameVerifierTest.java">Base | ||
| * revision</a> | ||
| */ | ||
| @Category({ MiscTests.class, SmallTests.class }) |
ContributorAuthor
There was a problem hiding this comment.
the test cases in this class were copied from zk, but there they use pre-generated certificate strings. Here I decided to use our existing X509TestHelpers to generate the certs at test time. I verified that the generated certs here match the expectation of the pre-generated ones in zk.
| * revision</a> | ||
| */ | ||
| @Category({ MiscTests.class, SmallTests.class }) | ||
| public class TestHBaseTrustManager { |
ContributorAuthor
There was a problem hiding this comment.
this class was copied verbatim
| } | ||
| @Test | ||
| public void testCreateSSLContextWithClientAuthDefault() throws Exception { |
ContributorAuthor
There was a problem hiding this comment.
I add tests for ClientAuth passthrough here, but it's not really possible (as far as i can tell) to access the TrustManager of the underlying SslContext/SSLEngine. So I can't easily add tests here for the TrustManager hostname verification arg passthrough. So instead we have the comprehensive end-to-end tests in TestMutualTlsClientSide/ServerSide
bbeaudreault
marked this pull request as ready for review
August 24, 2022 17:24
bbeaudreault
commented
Aug 24, 2022
ContributorAuthor
I've added comprehensive tests (about 250 test cases covering all permutations). I also marked with comments the files which are copied from ZK unchanged. |
bbeaudreault
commented
Aug 24, 2022
ContributorAuthor
@anmolnar I can't assign you as a reviewer, but would appreciate a look when you get a chance. |
Apache-HBase
commented
Aug 24, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Aug 24, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Aug 24, 2022
🎊 +1 overall
This message was automatically generated. |
bbeaudreault
commented
Sep 5, 2022
ContributorAuthor
@Apache9 any chance you can take a look at this? It's just a backport from zk like the other recent TLS patches, mostly copied except for the new end-to-end tests I added. |
anmolnar
approved these changes
Sep 7, 2022
anmolnar
left a comment
Contributor
There was a problem hiding this comment.
lgtm. Just a few nitpicks.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
anmolnar
commented
Sep 7, 2022
Contributor
@bbeaudreault One more thing before I forget: I think you don't need to include the documentation in this patch, but I suggest to create a separate ticket for it, rather than overloading my pending docs PR. Thanks. |
bbeaudreault
commented
Sep 7, 2022
ContributorAuthor
Yea, I was wondering we should add it to yours. I had started submitting a suggestion comment. But actually I think it might need a larger refactor to make it make sense relative to the "Secure Access" section. So I agree about creating a separate jira. |
bbeaudreaultforce-pushed
the
HBASE-27280
branch
from
September 8, 2022 21:31
4254782 to
4ef95b0CompareApache-HBase
commented
Sep 8, 2022
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 9, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 9, 2022
🎊 +1 overall
This message was automatically generated. |
ContributorAuthor
Yesterday I fixed the review feedback. Just pushed a commit to fix the checkstyle/errorprone failures -- I cant seem to run them locally, so we'll see. The spotbugs failure seems to both be wrong and unrelated to my change. |
Apache-HBase
commented
Sep 9, 2022
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 9, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 9, 2022
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 9, 2022
💔 -1 overall
This message was automatically generated. |
Apache9
commented
Sep 18, 2022
Contributor
Sorry, I brought my children to a tourist attraction and just got back. And I googled what does 'mutual' mean(sorry for why bad English...)... Now I know what is going on here. Let me take a look soon. |
Apache9
reviewed
Sep 18, 2022
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Apache9
reviewed
Sep 19, 2022
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Apache-HBase
commented
Sep 19, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 20, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 20, 2022
🎊 +1 overall
This message was automatically generated. |
Apache9
reviewed
Sep 20, 2022
Uh oh!
There was an error while loading. Please reload this page.
Apache-HBase
commented
Sep 20, 2022
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 20, 2022
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 20, 2022
💔 -1 overall
This message was automatically generated. |
bbeaudreault
commented
Sep 20, 2022
ContributorAuthor
Test failures are real but I ran out of time today. Will fix them up tomorrow morning |
Apache-HBase
commented
Sep 20, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 21, 2022
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 21, 2022
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 21, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 21, 2022
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 21, 2022
💔 -1 overall
This message was automatically generated. |
Apache9
reviewed
Sep 22, 2022
| if (type != null) { | ||
| if (type == SubjectName.DNS || type == SubjectName.IP) { | ||
| final Object o = entry.get(1); | ||
| // TODO ASN.1 DER encoded form when instanceof byte[] |
Contributor
There was a problem hiding this comment.
Will this be a follow on issue?
ContributorAuthor
There was a problem hiding this comment.
This todo existed in the file we pulled from ZK. I don't have plans to actually do that work unless we came up with some reason. I'll delete it
| hostAddress = inetAddress.getHostAddress(); | ||
| hostnameVerifier.verify(hostAddress, certificate); | ||
| } catch (SSLException addressVerificationException) { | ||
| if (!allowReverseDnsLookup) { |
Contributor
There was a problem hiding this comment.
For server side verification, if the InetAddress is created with a host name, we do not need to do a reverse dns lookup and usually we need to verify it through host name instead of ip address? So I prefer here we should try to find out whethter we already have a host name in the InetAddress, if so, just check it, otherwise we try to use reverse dns lookup, and the flag will take effect.
Contributor
There was a problem hiding this comment.
After googling, a possible way to check whether host name is present is to check the toString result, a bit hacky but could work.
/**
* Converts this IP address to a {@code String}. The
* string returned is of the form: hostname / literal IP
* address.
*
* If the host name is unresolved, no reverse name service lookup
* is performed. The hostname part will be represented by an empty string.
*
* @return a string representation of this IP address.
*/
public String toString() {
String hostName = holder().getHostName();
return ((hostName != null) ? hostName : "")
+ "/" + getHostAddress();
}
So we could check whether the toString result of the InetAddress starts with '/', if so, there is no host name provided, so if we want to check hostname, we need a reverse dns lookup, otherwise not.
Apache9
commented
Sep 22, 2022
Contributor
We are very close now, thanks @bbeaudreault for your patient! |
Apache9
approved these changes
Sep 22, 2022
bbeaudreault
commented
Sep 22, 2022
ContributorAuthor
Thanks for all the guidance and review here, Duo! |
bbeaudreaultforce-pushed
the
HBASE-27280
branch
from
September 22, 2022 15:41
c4726a9 to
4f8a369Comparebbeaudreaultforce-pushed
the
HBASE-27280
branch
from
September 22, 2022 15:42
4f8a369 to
de8ee7bComparebbeaudreault
commented
Sep 22, 2022
ContributorAuthor
Pre-commit is weirdly failing trying to checkout from git. I just squashed, rebased, and force pushed. No code changes. |
bbeaudreault
commented
Sep 22, 2022
ContributorAuthor
@Apache9 sorry, but any idea why this keeps continually failing?
I tried running that command locally and it works. The sha has changed every time i push, and the sha look correct. I wonder if i need to re-open this? |
Apache9
commented
Sep 23, 2022
Contributor
Not sure what's the problem. Let's close this PR and open a new one to see if it could fix the strange problem. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
As with the rest of the netty tls stuff, this is largely pulled and adapted from ZK. This adds support for setting netty's ClientAuth mode (none, need, want) which will handle the cert verification from trust store. We additionally extend the trust manager to enable verification of hostnames.
When TLS is enabled, the default will be all hostnames verified and ClientAuth.NEED. User can tune it down from there if desired.
Comes with comprehensive test of all possible modes from client and server side.
Note: There is follow-up work to be done in another issue which we can use the X509Certificate DN (distinguished name) to validate the ConnectionHeader's username and groups. This first pass simply plugs in the barebones mTLS support.
cc @anmolnar