HBASE-28921 Skip bundling hbase-webapps folder in jars by NihalJain · Pull Request #6368 · apache/hbase

NihalJain · 2024-10-17T12:36:01Z

We are bundling all webapp resources in hbase-server, hbase-thrift, hbase-rest and transitively to hbase-shaded-mapreduce jar. This can be an issue, say if any of the Js projects used by hbase are vulnerable, security scan tools like sonatype start flagging the jars too as vulnerable since they contain vulnerable code.

With this JIRA, we want to avoid bundling static webapp resources in our jars.

NihalJain · 2024-10-17T12:47:42Z

Built hbase locally, untarred and start master, rest and thrift server.

And then verified following:

Browse to master, rest and thrift web UIs and ensure no issues in web pages
Start hbase shell and do basic sanity
Run Thrift demo client from hbase-examples
Run basic REST APIs

Diff of the tarballs against master:

Apache-HBase · 2024-10-17T13:29:41Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 38s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ master Compile Tests _
+0 🆗	mvndep	0m 9s		Maven dependency ordering for branch
+1 💚	mvninstall	3m 3s		master passed
+1 💚	compile	4m 21s		master passed
+1 💚	spotless	0m 48s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 56s		the patch passed
+1 💚	compile	5m 11s		the patch passed
+1 💚	javac	5m 11s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	xmllint	0m 0s		No new issues.
+1 💚	hadoopcheck	13m 5s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚	spotless	0m 52s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 31s		The patch does not generate ASF License warnings.
		38m 56s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6368/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6368
Optional Tests	dupname asflicense javac codespell detsecrets xmllint hadoopcheck spotless compile
uname	Linux 33038e1321d8 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `8a073b1`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	85 (vs. ulimit of 30000)
modules	C: hbase-server hbase-thrift hbase-rest U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6368/1/console
versions	git=2.34.1 maven=3.9.8 xmllint=20913
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-10-17T17:07:59Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 14s		Docker mode activated.
-0 ⚠️	yetus	0m 2s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+0 🆗	mvndep	0m 10s		Maven dependency ordering for branch
+1 💚	mvninstall	3m 4s		master passed
+1 💚	compile	1m 49s		master passed
+1 💚	javadoc	1m 20s		master passed
+1 💚	shadedjars	5m 21s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 6s		the patch passed
+1 💚	compile	1m 47s		the patch passed
+1 💚	javac	1m 47s		the patch passed
+1 💚	javadoc	1m 20s		the patch passed
+1 💚	shadedjars	5m 20s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	217m 30s		hbase-server in the patch passed.
+1 💚	unit	7m 9s		hbase-thrift in the patch passed.
+1 💚	unit	3m 50s		hbase-rest in the patch passed.
		257m 19s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6368/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6368
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 9d01b49f6ef8 5.4.0-192-generic #212-Ubuntu SMP Fri Jul 5 09:47:39 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `8a073b1`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6368/1/testReport/
Max. process+thread count	5140 (vs. ulimit of 30000)
modules	C: hbase-server hbase-thrift hbase-rest U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6368/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

stoty · 2024-10-18T05:08:13Z

Are the webapps found in development mode (i.e. when starting HBase from the source directory) ?

NihalJain · 2024-10-18T05:35:33Z

Are the webapps found in development mode (i.e. when starting HBase from the source directory) ?

Should be available since we have following code at

hbase/bin/hbase

Line 204 in a8fbac6

if [ -d"$HBASE_HOME/hbase-server/target/hbase-webapps" ];then

:

#add if we are in a dev environment
if [ -d "$HBASE_HOME/hbase-server/target/hbase-webapps" ]; then
if [ "$COMMAND" = "thrift" ] ; then
add_to_cp_if_exists "${HBASE_HOME}/hbase-thrift/target"
elif [ "$COMMAND" = "thrift2" ] ; then
add_to_cp_if_exists "${HBASE_HOME}/hbase-thrift/target"
elif [ "$COMMAND" = "rest" ] ; then
add_to_cp_if_exists "${HBASE_HOME}/hbase-rest/target"
else
add_to_cp_if_exists "${HBASE_HOME}/hbase-server/target"
# Needed for GetJavaProperty check below
add_to_cp_if_exists "${HBASE_HOME}/hbase-server/target/classes"
fi
fi

Have never tried launching hbase from source directory. Let me try that as well.

NihalJain · 2024-10-18T05:41:38Z

Have never tried launching hbase from source directory. Let me try that as well.

Verified starting master, rest and thrift from source directory with this patch. All web UIs work fine.

hbase % pwd /Users/nihaljain/code/os/hbase
hbase % ./bin/hbase-daemon.sh start master
running master, logging to /Users/nihaljain/code/os/hbase/bin/../logs/hbase-nihaljain-master-HOSTNAME.out
hbase % ./bin/hbase-daemon.sh start rest running rest, logging to /Users/nihaljain/code/os/hbase/bin/../logs/hbase-nihaljain-rest-HOSTNAME.out
base % ./bin/hbase-daemon.sh start thrift
running thrift, logging to /Users/nihaljain/code/os/hbase/bin/../logs/hbase-nihaljain-thrift-HOSTNAME.out

stoty

+1 LGTM

NihalJain · 2024-10-18T11:24:55Z

Thank you @stoty for reviewing. I will merge this to branch-2+ by EOD.

stoty · 2024-10-18T12:19:07Z

Since this is a security imrovement, I would consider merging this to all active branches.

NihalJain · 2024-10-18T14:37:06Z

Since this is a security imrovement, I would consider merging this to all active branches.

Actually, you are right. Will push to all active branches.

We are bundling all webapp resources in hbase-server, hbase-thrift, hbase-rest and transitively to hbase-shaded-mapreduce jar. This can be an issue, say if any of the Js projects used by hbase are vulnerable, security scan tools like sonatype start flagging the jars too as vulnerable since they contain vulnerable code. With this JIRA, we skip bundling static webapp resources in our jars. Signed-off-by: Istvan Toth <stoty@apache.org> Reviewed-by: Dávid Paksy <paksyd@cloudera.com> (cherry picked from commit 8366304)

NihalJain · 2024-10-21T16:25:41Z

Hi all, thank you for your reviews, merged to master. Also, raised backport for all active branches.

Hi @chandrasekhar-188k request you to update your email id on your github profile, so that can give you due credit for your reviews during PR merge.

This reverts commit 8366304.

HBASE-28921 Skip bundling hbase-webapps folder in jars
8a073b1

stoty approved these changes Oct 18, 2024
View reviewed changes

chandrasekhar-188k approved these changes Oct 20, 2024
View reviewed changes

PDavid approved these changes Oct 21, 2024
View reviewed changes

NihalJain merged commit 8366304 into apache:masterOct 21, 2024

NihalJain mentioned this pull request Oct 21, 2024
HBASE-28921 Skip bundling hbase-webapps folder in jars (#6368) #6376
Closed

NihalJain mentioned this pull request Oct 21, 2024
HBASE-28921 Skip bundling hbase-webapps folder in jars (#6368) #6377
Closed

NihalJain mentioned this pull request Oct 21, 2024
HBASE-28921 Skip bundling hbase-webapps folder in jars (#6368) #6378
Closed

NihalJain mentioned this pull request Oct 21, 2024
HBASE-28921 Skip bundling hbase-webapps folder in jars (#6368) #6379
Closed

NihalJain added a commit that referenced this pull request Oct 21, 2024
Revert "HBASE-28921Skip bundling hbase-webapps folder in jars (#6368)"
ece0fdb
This reverts commit 8366304.

NihalJain mentioned this pull request Oct 21, 2024
Revert "HBASE-28921 Skip bundling hbase-webapps folder in jars" #6382
Merged

NihalJain added a commit that referenced this pull request Oct 21, 2024
Revert "HBASE-28921Skip bundling hbase-webapps folder in jars (#6368)"
67f01c5
This reverts commit 8366304.

NihalJain mentioned this pull request Oct 23, 2024
HBASE-28921 Avoid bundling hbase-webapps folder in default jars #6388
Merged

Navigation Menu

Uh oh!

HBASE-28921 Skip bundling hbase-webapps folder in jars - #6368

HBASE-28921 Skip bundling hbase-webapps folder in jars#6368
NihalJain merged 1 commit into
apache:masterfrom
NihalJain:HBASE-28921

NihalJain commented Oct 17, 2024

Uh oh!

NihalJain commented Oct 17, 2024

Uh oh!

Apache-HBase commented Oct 17, 2024

Uh oh!

Apache-HBase commented Oct 17, 2024

Uh oh!

stoty commented Oct 18, 2024

Uh oh!

NihalJain commented Oct 18, 2024

Uh oh!

NihalJain commented Oct 18, 2024

Uh oh!

stoty left a comment

Uh oh!

NihalJain commented Oct 18, 2024

Uh oh!

stoty commented Oct 18, 2024

Uh oh!

NihalJain commented Oct 18, 2024•
edited
Loading

Uh oh!

NihalJain commented Oct 21, 2024•
edited
Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

NihalJain commented Oct 17, 2024

Uh oh!

NihalJain commented Oct 17, 2024

Uh oh!

Apache-HBase commented Oct 17, 2024

Uh oh!

Apache-HBase commented Oct 17, 2024

Uh oh!

stoty commented Oct 18, 2024

Uh oh!

NihalJain commented Oct 18, 2024

Uh oh!

NihalJain commented Oct 18, 2024

Uh oh!

stoty left a comment

Choose a reason for hiding this comment

Uh oh!

NihalJain commented Oct 18, 2024

Uh oh!

stoty commented Oct 18, 2024

Uh oh!

NihalJain commented Oct 18, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NihalJain commented Oct 21, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

NihalJain commented Oct 18, 2024•
edited
Loading

NihalJain commented Oct 21, 2024•
edited
Loading