STORM-517: Adding JAVA_SERVICE_NAME to bin/storm

[solarce]

STORM-517: Adding JAVA_SERVICE_NAME to bin/storm

• Adds reading an JAVA_SERVICE_NAME environment
  variable to bin/storm and reads it in as part of
  generating all_args in exec_storm_class()

Meant to be set to something like:

export JAVA_SERVICE_NAME="-Dservice=storm-nimbus'

See https://github.com/solarce/storm-cookbook/blob/master/templates/default/storm-upstart-conf.erb#L19 for an example with Chef

[harshach]

@solarce any reason not to pass this value from nimbus , supervisor etc methods in storm command.
Having it as environment variable is another step for user to enable it.

[nathanmarz]

-1

I've never heard of this environment variable before. What is it used for?

Also, you can get these options added to the Java processes already by making use of nimbus.childopts, supervisor.childopts, and worker.childopts, so it's not clear why Storm needs specialized support for this variable.

[solarce]

@nathanmarz

It's not any existing environment variable, it's just a name I made up.

I do use nimbus.childops, etc for adding JMX related arguments because their position in the process arguments doesn't matter. As to why I don't just use nimbus.childopts, etc for this, I'm copying my reasoning from what I wrote in https://issues.apache.org/jira/browse/STORM-517.

Currently the way that bin/storm starts the various storm processes (nimbus, ui, supervisor) results in a process list entry like the following, for each process:

ubuntu 21940 0.4 3.9 2294904 151384 ? Ssl Oct03 2:47 java -server -Dstorm.options= -Dstorm.home=/home/ubuntu/apache-storm-0.9.2-incubating -Djava.library.path=/usr/local/lib -Dstorm.conf.file= -cp /home/ubuntu/apache-storm-0.9.2-incubating/lib/asm-4.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/jline-2.11.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/guava-13.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/ubuntu/apache-storm-0.9.2-incubating/conf -Xmx1024m -Dlogfile.name=nimbus.log -Dlogback.configurationFile=/home/ubuntu/apache-storm-0.9.2-incubating/logback/cluster.xml backtype.storm.daemon.nimbus

This means that it can be difficult to differentiate between the processes in the process list and monitoring tools that use SNMP to get the process list (such as Zenoss) can't differentiate because the process list arguments gets truncated way before the list of .jars in the text stops being the same.

Ideally we'd be able to set -Dservice=foo in bin/storm via an environment variable, so it's easy to set with things like upstart, runit, etc

I realize that the SNMP truncation of the process arguments is a factor of my environment, but it's not an uncommon way to do process monitoring, and I felt that adding this environment variable that's optional was the least intrusive way to add it.

If there is another way for me to get -Dservice= at the beginning of the process arguments with the existing bin/storm, I'd love to hear about it

[nathanmarz]

I see. I'd be fine with a different patch which does not use any environment vars and instead sets -Dstorm.service= to be one of "nimbus", "supervisor", or "worker", passed appropriately in the bin/storm script or where the supervisor launches workers in the code

[solarce]

That makes sense, I'll rework it and test that later this week

[revans2]

@solarce any update on this?

[d2r]

@solarce I think this was taken care of as part of STORM-833: https://github.com/apache/storm/blame/f75cf7c176bf5cae740c2d29999d57de87ea07fa/bin/storm.py#L187

[d2r]

@solarce This appears to have been addressed as part of #562 . We should probably close this pull request.