Skip to content

HBASE-26553 OAuth Bearer authentication mech plugin for SASL - #4733

Closed
anmolnar wants to merge 5 commits into
apache:masterfrom
anmolnar:HBASE-26553_rebase
Closed

HBASE-26553 OAuth Bearer authentication mech plugin for SASL#4733
anmolnar wants to merge 5 commits into
apache:masterfrom
anmolnar:HBASE-26553_rebase

Conversation

@anmolnar

@anmolnaranmolnar commented Aug 26, 2022•
edited
Loading

Copy link
Copy Markdown
Contributor

Adds a new SASL mech plugin for OAuthBearer (JWT) authentication.

  • In order to keep the size of this initial patch manageable, the supported workflow is limited: client reads a single JWT token with expiry information from environment variable and authenticates with the server.
  • It works similarly to Hadoop delegation tokens, JWT token takes precedence, but if it's missing, the auth provider will fall back to Kerberos.
  • Kerberos must be enabled on the cluster, otherwise HBase security is not enabled.

Minimum configuration to enable JWT auth:

Server side:

 <property>
<name>hbase.server.sasl.provider.extras</name>
<value>org.apache.hadoop.hbase.security.provider.OAuthBearerSaslServerAuthenticationProvider</value>
</property>
<property>
<name>hbase.security.oauth.jwt.jwks.url</name>
<value>JWKS download url</value>
</property>

Client side:

 <property>
<name>hbase.client.sasl.provider.extras</name>
<value>org.apache.hadoop.hbase.security.provider.OAuthBearerSaslClientAuthenticationProvider</value>
</property>
<property>
<name>hbase.client.sasl.provider.class</name>
<value>org.apache.hadoop.hbase.security.provider.OAuthBearerSaslProviderSelector</value>
</property>

The client also has to be in possession of a valid JWT token which must set via environment variable:

export HBASE_JWT="<base64 encoded token>,<expiry>"

cc @petersomogyi@meszibalu@joshelser@bbeaudreault@Apache9

@anmolnar
HBASE-26655. Implement new OAuth Bearer SASL plugin
893f392
Signed-off-by: Josh Elser <elserj@apache.org>
@anmolnar
HBASE-26665 Added e2e test with HBase mini-cluster (apache#4051)
5bb35dd
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Peter Somogyi <psomogyi@apache.org>
@anmolnar
HBASE-26667 Integrate user-experience for hbase-client (apache#4064)
1532e05
Signed-off-by: Josh Elser <elserj@apache.org>
@anmolnar
HBASE-26553. Move JWKS loader to upper level, fix dependencies
24d0ba0
@Apache-HBase

Apache-HBase commented Aug 26, 2022

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 2sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗mvndep0m 18sMaven dependency ordering for branch
+1 💚mvninstall2m 24smaster passed
+1 💚compile6m 12smaster passed
+1 💚checkstyle1m 1smaster passed
+1 💚spotless0m 41sbranch has no errors when running spotless:check.
+1 💚spotbugs10m 25smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 16sMaven dependency ordering for patch
+1 💚mvninstall2m 13sthe patch passed
+1 💚compile6m 11sthe patch passed
-0 ⚠️javac6m 11sroot generated 3 new + 707 unchanged - 0 fixed = 710 total (was 707)
-0 ⚠️checkstyle1m 0sroot: The patch generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚xml0m 1sThe patch has no ill-formed XML file.
+1 💚hadoopcheck8m 0sPatch does not cause any errors with Hadoop 3.2.4 3.3.4.
-1 ❌spotless0m 12spatch has 71 errors when running spotless:check, run spotless:apply to fix.
+1 💚spotbugs10m 51sthe patch passed
_ Other Tests _
+1 💚asflicense0m 58sThe patch does not generate ASF License warnings.
58m 2s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#4733
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile xml
unameLinux 6546613b29b7 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 06728e5
Default JavaAdoptOpenJDK-1.8.0_282-b08
javachttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/1/artifact/yetus-general-check/output/diff-compile-javac-root.txt
checkstylehttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/1/artifact/yetus-general-check/output/diff-checkstyle-root.txt
spotlesshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/1/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count138 (vs. ulimit of 30000)
modulesC: hbase-common hbase-client hbase-resource-bundle hbase-server hbase-examples . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/1/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Aug 26, 2022

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 53sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 21sMaven dependency ordering for branch
+1 💚mvninstall2m 41smaster passed
+1 💚compile1m 43smaster passed
+1 💚shadedjars3m 58sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc2m 54smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 14sMaven dependency ordering for patch
+1 💚mvninstall2m 31sthe patch passed
+1 💚compile1m 43sthe patch passed
+1 💚javac1m 43sthe patch passed
+1 💚shadedjars3m 54spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc2m 55sthe patch passed
_ Other Tests _
-1 ❌unit213m 42sroot in the patch failed.
239m 30s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#4733
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 5c8a411d2553 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 06728e5
Default JavaAdoptOpenJDK-11.0.10+9
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/1/testReport/
Max. process+thread count2599 (vs. ulimit of 30000)
modulesC: hbase-common hbase-client hbase-resource-bundle hbase-server hbase-examples . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Aug 26, 2022

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 56sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 26sMaven dependency ordering for branch
+1 💚mvninstall3m 7smaster passed
+1 💚compile2m 2smaster passed
+1 💚shadedjars5m 38sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc3m 24smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 15sMaven dependency ordering for patch
+1 💚mvninstall2m 59sthe patch passed
+1 💚compile2m 12sthe patch passed
+1 💚javac2m 12sthe patch passed
+1 💚shadedjars5m 38spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc3m 42sthe patch passed
_ Other Tests _
+1 💚unit408m 38sroot in the patch passed.
442m 55s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#4733
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 7c2a7e7144fd 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 06728e5
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/1/testReport/
Max. process+thread count4855 (vs. ulimit of 30000)
modulesC: hbase-common hbase-client hbase-resource-bundle hbase-server hbase-examples . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/1/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Aug 28, 2022

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 26sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗mvndep0m 21sMaven dependency ordering for branch
+1 💚mvninstall3m 27smaster passed
+1 💚compile9m 43smaster passed
+1 💚checkstyle1m 22smaster passed
+1 💚spotless0m 57sbranch has no errors when running spotless:check.
+1 💚spotbugs15m 2smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 16sMaven dependency ordering for patch
+1 💚mvninstall3m 19sthe patch passed
+1 💚compile8m 46sthe patch passed
-0 ⚠️javac8m 46sroot generated 3 new + 707 unchanged - 0 fixed = 710 total (was 707)
+1 💚checkstyle1m 19sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚xml0m 2sThe patch has no ill-formed XML file.
+1 💚hadoopcheck10m 55sPatch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚spotless0m 52spatch has no errors when running spotless:check.
+1 💚spotbugs15m 57sthe patch passed
_ Other Tests _
+1 💚asflicense0m 52sThe patch does not generate ASF License warnings.
82m 31s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#4733
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile xml
unameLinux b3cdeb7656d8 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 37651ee
Default JavaAdoptOpenJDK-1.8.0_282-b08
javachttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/2/artifact/yetus-general-check/output/diff-compile-javac-root.txt
Max. process+thread count138 (vs. ulimit of 30000)
modulesC: hbase-common hbase-client hbase-resource-bundle hbase-server hbase-examples . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/2/console
versionsgit=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Aug 28, 2022

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 41sDocker mode activated.
-0 ⚠️yetus0m 2sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 20sMaven dependency ordering for branch
+1 💚mvninstall2m 40smaster passed
+1 💚compile1m 47smaster passed
+1 💚shadedjars3m 53sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc2m 53smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 14sMaven dependency ordering for patch
+1 💚mvninstall2m 27sthe patch passed
+1 💚compile1m 43sthe patch passed
+1 💚javac1m 43sthe patch passed
+1 💚shadedjars3m 53spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc2m 54sthe patch passed
_ Other Tests _
+1 💚unit249m 44sroot in the patch passed.
276m 22s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#4733
Optional Testsjavac javadoc unit shadedjars compile
unameLinux b765cb021a83 5.4.0-1081-aws #88~18.04.1-Ubuntu SMP Thu Jun 23 16:29:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 37651ee
Default JavaAdoptOpenJDK-11.0.10+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/2/testReport/
Max. process+thread count4757 (vs. ulimit of 30000)
modulesC: hbase-common hbase-client hbase-resource-bundle hbase-server hbase-examples . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Aug 28, 2022

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 19sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 18sMaven dependency ordering for branch
+1 💚mvninstall2m 21smaster passed
+1 💚compile1m 36smaster passed
+1 💚shadedjars3m 46sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc2m 38smaster passed
_ Patch Compile Tests _
+0 🆗mvndep0m 14sMaven dependency ordering for patch
+1 💚mvninstall2m 14sthe patch passed
+1 💚compile1m 37sthe patch passed
+1 💚javac1m 37sthe patch passed
+1 💚shadedjars3m 45spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc2m 39sthe patch passed
_ Other Tests _
+1 💚unit376m 13sroot in the patch passed.
400m 5s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#4733
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 398b44e09ce9 5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 37651ee
Default JavaAdoptOpenJDK-1.8.0_282-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/2/testReport/
Max. process+thread count4614 (vs. ulimit of 30000)
modulesC: hbase-common hbase-client hbase-resource-bundle hbase-server hbase-examples . U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4733/2/console
versionsgit=2.17.1 maven=3.6.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

taklwu
taklwu reviewed Aug 30, 2022
View reviewed changes

@taklwutaklwu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall if we can reused the reviews in HBASE-26655, HBASE-26665, HBASE-26667 that merged into the feature branch of HBASE-26553, this is just the same commits with minor fixed for the loadJwkSet(). But if you're looking a new round of reviews, please let me know.

below are two minor comments that isn't a blocker for this PR.

  1. [nit] HBASE-26655, HBASE-26665, HBASE-26667 was reviewed as part of the feature HBASE-26553 , but why HBASE-26655 does not have a link to #4019 ?
  2. [nit] for the minimum configuration mentioned in the description, don't we also need to configure the environment parameter of HBASE_JWT for the client? maybe we should tell the minimum configuration for the client and server separately in the doc ?

@anmolnar

anmolnar commented Sep 12, 2022

Copy link
Copy Markdown
ContributorAuthor

Thanks @taklwu , I've updated the minimum configuration in the description with more details.

Sorry for the confusion, I've decided to abandon the feature branch to speed up the rebasing and the process. Please let me know if you think it would be better to go back and continue working on the feature branch instead.

@joshelser

but why HBASE-26655 does not have a link to #4019 ?

I see the link in the jira.

@anmolnar
anmolnar deleted the HBASE-26553_rebase branch April 28, 2023 10:31
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@taklwutaklwutaklwu left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@anmolnar@Apache-HBase@taklwu