Embedded jetty inside shaded jar by marcaurele · Pull Request #2207 · apache/cloudstack

marcaurele · 2017-07-28T06:21:33Z

This PR is to remove the dependency on distro Tomcat to ship CloudStack directly with an embedded Jetty. The delivery will only be scripts, one fat jar, JCE jar and default properties files.

Remove Tomcat from requirement
Embed Jetty
Shaded jar
Modify init script (classpath must contain bouncycastle) + log4j & properties files)

marcaurele · 2017-07-28T06:22:25Z

@rhtyd I still have to add a couple of things. There isn't anymore any JSP for the UI, right?

wido · 2017-07-28T06:40:05Z

+ public void init(DaemonContext context) {
+ Properties props = System.getProperties();
+ setPort(Integer.parseInt(props.getProperty("jetty.port", "8080")));
+ setBindInterface(props.getProperty("jetty.host"));


To what does jetty.host default to? Reason I'm asking is IPv6. Just want to make sure we bind on '::' and not 0.0.0.0 so that IPv6 works out of the box

Good point @wido but it should work since if there isn't any value it's binding to * which should includes IPv6, right?

wido · 2017-07-28T06:40:30Z

Very nice! Very welcome.

I think we also need a modified systemd service to make this work, right?

marcaurele · 2017-07-28T07:04:26Z

@wido yes, it's one of the bullet in the list still open. I have to copy paste what I've done and adapt as much as I can for the upstream version. It's really a first shot to let @rhtyd know what I haven't forgotten to work on it. More to come and I'll ping when it's ready for testing/adaptation

yadvr · 2017-07-28T08:16:53Z

@marcaurele thanks yes, JSPs have been removed from UI now. Good to see this, I'll close my PR.
+1 to your list, however for the first release we may do without having a single fat/shaded jar but instead an exploded war might also work. A lot of people might want to access assets and other files (especially UI files, for example to enable/disable UI plugins by editing js files etc). And, we also want to be able to configure and support SSL certificates with jetty.

marcaurele · 2017-08-07T08:08:05Z

@rhtyd I think I backported all necessary changes (for Ubuntu distro). Can you have a look & try?

yadvr · 2017-08-07T09:35:58Z

@marcaurele thanks, I'll have a look this week. Can you fix Travis failures as well, are there any changes wrt how a developer builds, runs and tests CloudStack?

I've one reservation -- if we put everything in a single jar, it might be difficult for users to apply UI changes and add custom jars, anyway to do that? Maybe add a custom path (add a classpath) for an override directory from where jars may be loaded and maybe split the UI as a separate rpm/deb?

Can you also allow me to push on the exoscale:CLOUDSTACK-10012 branch? Alternatively, you may push the changes to the upstream branch so I can collaborate and make changes with you.

yadvr · 2017-08-07T09:37:13Z

@blueorangutan package

blueorangutan · 2017-08-07T09:38:10Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

wido · 2017-08-07T09:55:13Z

I see you modified the init script, but as Ubuntu and CentOS have moved to systemd, shouldn't we fix that as well?

yadvr · 2017-08-07T10:08:42Z

@wido that should, however testing is required. I see apidocs build is broken that blocks us from further testing. /cc @marcaurele

blueorangutan · 2017-08-07T10:22:01Z

Packaging result: ✖centos6 ✖centos7 ✖debian. JID-925

yadvr · 2017-08-07T11:22:39Z

@marcaurele I've found the fix, can you put this commit in your branch as I cannot push changes:

diff --git a/tools/apidoc/build-apidoc.sh b/tools/apidoc/build-apidoc.sh
index 8ab69c8373..4e1f9a639d 100755
--- a/tools/apidoc/build-apidoc.sh
+++ b/tools/apidoc/build-apidoc.sh
@@ -47,7 +47,7 @@ fi
CP=$PATHSEP/
-java -cp $CP$PATHSEP$TARGETJARDIR/*$PATHSEP$DEPSDIR/* com.cloud.api.doc.ApiXmlDocWriter -d "$DISTDIR" $*
+java -cp $CP$PATHSEP$TARGETJARDIR$PATHSEP$DEPSDIR com.cloud.api.doc.ApiXmlDocWriter -d "$DISTDIR" $*
if [ $? -ne 0 ]
then
diff --git a/tools/apidoc/gen_toc.py b/tools/apidoc/gen_toc.py
index a261037d80..39e1576422 100644
--- a/tools/apidoc/gen_toc.py
+++ b/tools/apidoc/gen_toc.py
@@ -180,7 +180,8 @@ known_categories = {
'stopNetScalerVpx' : 'Load Balancer',
'deleteServicePackageOffering' : 'Load Balancer',
'destroyNsVpx' : 'Load Balancer',
- 'startNsVpx' : 'Load Balancer'
+ 'startNsVpx' : 'Load Balancer',
+ 'listElastistorInterface': 'Misc'
}
diff --git a/tools/apidoc/pom.xml b/tools/apidoc/pom.xml
index e6804a0e78..61c1ccd593 100644
--- a/tools/apidoc/pom.xml
+++ b/tools/apidoc/pom.xml
@@ -21,8 +21,8 @@
<relativePath>../pom.xml</relativePath>
</parent>
<properties>
- <client.config.base>../../client/target/cloud-client-ui-${project.version}/WEB-INF/</client.config.base>
- <client.config.jars>${client.config.base}/lib</client.config.jars>
+ <client.config.base>../../client/target/</client.config.base>
+ <client.config.jars>${client.config.base}/cloud-client-ui-${project.version}.jar</client.config.jars>
<client.config.conf>${client.config.base}/classes</client.config.conf>
</properties>

yadvr · 2017-08-07T12:36:36Z

@marcaurele with the above fix I could get the build to work, however can you share how would you run the management server both using mvn and scripts? I'm unable to get it to work.

marcaurele · 2017-08-07T13:43:25Z

Even if it looks like a fat/uber jar, the startup script still runs it as a normal multi jar. If you really want, you could have a single jar. Due to bouncycastle and the JCE signature check, it is not possible to run it as a single jar.

To start it, I use:

# normal start
java -Dlog4j.configuration=log4j-cloud.xml -cp "client/target/conf:client/target/bcprov-jdk15on-1.55.jar:client/target/cloud-client-ui-X.X.X.X-SNAPSHOT.jar" org.apache.cloudstack.ServerDaemon
# with remote debug for example and specifying a port for Jetty
java -Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n -Djava.net.preferIPv4Stack=true -Dlog4j.configuration=log4j-cloud.xml -Djetty.port=8080 -cp "client/target/conf:client/target/bcprov-jdk15on-1.55.jar:client/target/cloud-client-ui-X.X.X.X-SNAPSHOT.jar" org.apache.cloudstack.ServerDaemon

marcaurele · 2017-08-07T13:46:11Z

+# Load extra arguments from java-opts.conf if any
+if [ -r "$MANAGEMENT_CONFIG/java-opts.conf" ]; then
+	JAVA_EXTRA_OPTS=$(cat $MANAGEMENT_CONFIG/java-opts.conf)
+	JAVA_OPTS="$JAVA_OPTS $JAVA_EXTRA_OPTS"


@rhtyd there's the inclusion of a java-opts.conf file if present that helps to add extra parameters to the classpath, like extra JARs, extra config directories, JMX...

marcaurele · 2017-08-07T13:49:12Z

There should be an option to let maintainer edit a PR (on github), so I guess the content of it too. But it does not seem to be active on the repository (https://help.github.com/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork/)

yadvr · 2017-08-07T14:32:27Z

@marcaurele thanks, I started working on this PR based on your branch to fix the centos7 issues: #2226 this is on ASF remote so any committer should be able to commit or send PRs.

marcaurele · 2017-08-07T14:47:52Z

Ok, closing this one then in favor of #2226

yadvr · 2017-08-07T14:50:56Z

Thanks @marcaurele please collaborate on #2226 , feel free to directly push changes. I'll use your suggestions and incorporate them in the systemd scripts now, currently trying to fix centos packaging.

marcaurele changed the title ~~[WIP] Inception for backport jetty embedded with shaded jar~~[WIP] Embedded jetty inside shaded jarJul 28, 2017

Inception for backport jetty embedded with shaded jar
a576e61

marcaurele force-pushed the CLOUDSTACK-10012 branch from 75a22f2 to a576e61Compare July 28, 2017 06:34

wido added the component:management-server label Jul 28, 2017

wido reviewed Jul 28, 2017
View reviewed changes

yadvr added component:packaging component:systemd status:work-in-progress labels Jul 28, 2017

yadvr mentioned this pull request Jul 28, 2017
CLOUDSTACK-10012: Embedded Jetty for CloudStack #2197
Closed

marcaurele added 3 commits July 31, 2017 08:11

Modify cloudtack-amanement init with uber jar startup
bb8fe4b

Remove dependency on tomcat, modify installation script for debian
a21588a

Add jetty dependencies version 9.2 (java 7+)
375eb72

marcaurele force-pushed the CLOUDSTACK-10012 branch from 717792e to 375eb72Compare August 3, 2017 05:26

marcaurele changed the title ~~[WIP] Embedded jetty inside shaded jar~~Embedded jetty inside shaded jarAug 7, 2017

yadvr mentioned this pull request Aug 7, 2017
CLOUDSTACK-10012: Towards CloudStack fat jar #2226
Merged
12 tasks

Fix for apidoc generation thanks to @rhtyd
931ed64

marcaurele commented Aug 7, 2017
View reviewed changes

marcaurele closed this Aug 7, 2017

Uh oh!

Conversation

marcaurele commented Jul 28, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

marcaurele commented Jul 28, 2017

Uh oh!

widoJul 28, 2017

Choose a reason for hiding this comment

Uh oh!

marcaureleJul 28, 2017

Choose a reason for hiding this comment

Uh oh!

wido commented Jul 28, 2017

Uh oh!

marcaurele commented Jul 28, 2017

Uh oh!

yadvr commented Jul 28, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

marcaurele commented Aug 7, 2017

Uh oh!

yadvr commented Aug 7, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yadvr commented Aug 7, 2017

Uh oh!

blueorangutan commented Aug 7, 2017

Uh oh!

wido commented Aug 7, 2017

Uh oh!

yadvr commented Aug 7, 2017

Uh oh!

blueorangutan commented Aug 7, 2017

Uh oh!

yadvr commented Aug 7, 2017

Uh oh!

yadvr commented Aug 7, 2017

Uh oh!

marcaurele commented Aug 7, 2017

Uh oh!

marcaureleAug 7, 2017

Choose a reason for hiding this comment

Uh oh!

yadvrAug 7, 2017

Choose a reason for hiding this comment

Uh oh!

marcaurele commented Aug 7, 2017

Uh oh!

yadvr commented Aug 7, 2017

Uh oh!

marcaurele commented Aug 7, 2017

Uh oh!

yadvr commented Aug 7, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

marcaurele commented Jul 28, 2017•
edited
Loading

yadvr commented Jul 28, 2017•
edited
Loading

yadvr commented Aug 7, 2017•
edited
Loading