ZEPPELIN-46: set only non-empty values for for spark.* properties

[bzz]

https://issues.apache.org/jira/browse/ZEPPELIN-46

[bzz]

So far this does not have any tests.
Thoughts on how\where to test this are warmly welcome!

[jongyoul]

@bzz Don't you mean !key.startsWith("spark.")?

[bzz]

@jongyoul indeed! Thank you.
How can we test this except extracting a new helper method and testing it separately?

[jongyoul]

@bzz Simply add this test in SparkInterpreterTest.java

@Test
public void testSparkConfCorrect() {
for (Tuple2<String, String> tuple2 : repl.getSparkContext().getConf().getAll()) {
if (tuple2._1().startsWith("spark."))
assertFalse(String.format("configuration starting from 'spark.' should not be empty. [%s]: [%s]", tuple2._1(), tuple2._2()), tuple2._2().isEmpty());
}
}

[jongyoul]

@bzz My test is not perfect. I'll change it

[jongyoul]

@bzz I've found strange thing while I'm changing my test cases, why do you know intpProperty's size is 0 if I'm running debugging?

And this is a live screenshot for sparkConf. It works fine but I don't know why something's wrong.

EDIT:
I've found the reason and make a PR #41

[felixcheung]

what about if val is whitespace? eg. " "

[jongyoul]

@felixcheung trim() will make whitespace removed. Tests case is not covered all of cases. I make a PR of @bzz and he will fix this.

[bzz]

Then val.trim().isEmpty() is true, so it will be set for any property, that does not start with "spark." (i.e whitespace as a password)

[jongyoul]

@bzz Sorry, I didn't reformat this. This needs one space between 'if' and '('.

[bzz]

Ready to merge

[felixcheung]

LGTM

[syepes]

I have just applied this patch and there still seems to be an issue.
If I change the property "spark.executor.memory = 20g" and restart Zeppelin with debugging, It still uses the default value of "512m" knowing that in the interpreter.json file its set to 20g

[Leemoonsoo]

Tested and LGTM.

for the problem @syepes mentioned, I have created fix #42

[bzz]

Merging if there is no discussion