STORM-904: Move bin/storm command line to java.

[priyank5485]

No description provided.

[priyank5485]

This PR aims at factoring out the logic for storm command line client which currently resides in two places. One in storm bash script and storm.py for Unix and storm.cmd for Windows. Idea is to create a java program that will be called from storm.cmd and storm bash script which will have all the logic from storm.py and storm.cmd in one place. This will help us maintain the storm client better since we will have only one file to change and in most cases only the base class that both Windows and Unix classes inherit from. I have already removed storm.py. I am working on finishing up the java file and also removing the storm.cmd file. Just putting it out there to elicit response to this approach from the community.

[caofangkun]

If run on Windows ,do nothing but return ？

[priyank5485]

@caofangkun thanks for checking this out. The code is still not complete. I need to put the windows implementation as well. Most of the code in the UnixStormCommandExecutor will move to abstract base class so that WindowsStormCommandLineExecutor can reuse. It will only have code that is specific to windows.

[knusbaum]

I have a little confusion about why we're moving the python to java instead of just having the storm.cmd run the existing storm.py.

I'm all in favor of de-duplicating the storm.py and storm.cmd commands, but I'd rather reuse storm.py than reimplementing it.

[harshach]

@knusbaum it will make lot easier for commands to be in java and we can have just wrappers for windows & nix* os. We don't need to have python dependency and in our experience of running python to execute java command to start a jvm takes lot of time and in larger clusters Ambari times out these commands.

[ptgoetz]

Any update on this PR? It would seem it at least needs an upmerge and additional review.

[hustfxj]

not all process need waitfor(). maybe you can add option. Like this:

public static java.lang.Process launch_process(final String command, final Map<String, String> environment, boolean backend) throws IOException {
if (backend == true) {
new Thread(new Runnable() {
@Override
public void run() {
String[] cmdlist = (new String("nohup " + command + " &")).split(" ");
try {
launchProcess(cmdlist, environment);
} catch (IOException e) {
LOG.error("Failed to run " + command + ":" + e.getCause(), e);
}
}
}).start();
return null;
} else {
String[] cmdlist = command.split(" ");
return launchProcess(cmdlist, environment);
}
}

[longdafeng]

@priyank5485

I don't prefer switch storm.py from python to java. Because I often add debug point in the storm.py, it is very easy to debug in online system(especially when environment is bad or jvm parameter is wrong). if we change to java version, it is very hard to debug.

Another point is that python's code is more easy to add new functions in the script.

[priyank5485]

@hustfxj I might have missed something and its been a long time since i did this. I could not find any storm command that would not need a waitFor. Can you give an example? If we do need that flag it should be easy to add and I can change that.

@longdafeng I am not sure I understand the point about debugging. You can still put a break point in this java version of storm command line. I agree with you that its easier to write new functions in script. But the whole point of this JIRA is to have one client that works for Unix based systems and Windows platform. The last time I was working on it we had storm.py for Unix and storm batch client for windows. Both of them also had a separate file for environment variables. It is hard to maintain two scripts anytime we want to change the storm command. We can ask others about their opinion and what they think is better between the two approaches.

[priyank5485]

@longdafeng Also please see one of the previous comments about why not to use python.