[STORM-833] Logging framework logback -> log4j 2.x

[kishorvpatil]

As part of changes to logging framework:

• Migrate from logback to log4j2
• Use RFC5424 foramt for routing logs to rsyslog
• Get stdout and stderr logged and rolled
• Add username and sensitivity to rsyslog RFC524 message format.

[revans2]

For a first pass things look good. I am +1, but I wrote some of the code you cherry picked in so my +1 only applies to code I didn't touch :)

[d2r]

Seems with this change the worker will also redact the ZooKeeper authentication payload from the conf, so if it is defined, then the secret is not logged. This is a good additional improvement.

I built the code, ran the tests, and checked that logging was happening with the ExclamationTopology.

I am +1

[HeartSaVioR]

We can see Travis CI is complaining that log is exceed 4M again.
https://travis-ci.org/apache/storm/builds/65275059
Note that Travis CI notifies ERROR, not FAIL. It's not random test failures so we shouldn't ignore it.
(Same icon - X - but icon's tooltip is different.)

I filed issue here - https://issues.apache.org/jira/browse/STORM-850.

To prevent it, we used logback-test.xml from storm-core, and it doesn't converted to log4j2-test.xml.
https://github.com/apache/storm/blob/master/storm-core/test/resources/logback-test.xml

And I can find how to use system environment from config file, but I can't find how to set default value.
It is important since we only need to change log level when running with Travis CI.

I agree its limitation is bad, and current approach may not be best.

[revans2]

Sorry about that I completely forgot about that part. I will try to get a pull request up for it today.

[HeartSaVioR]

No problem. Seems like random test failures make us not checking Travis CI's result.
We should address Kafka random test failure so that we are about to be sure we should take an action when Travis CI complains something is wrong.