[SPARK-20887][CORE] support alternative keys in ConfigBuilder

[cloud-fan]

What changes were proposed in this pull request?

ConfigBuilder builds ConfigEntry which can only read value with one key, if we wanna change the config name but still keep the old one, it's hard to do.

This PR introduce ConfigBuilder.withAlternative, to support reading config value with alternative keys. And also rename spark.scheduler.listenerbus.eventqueue.size to spark.scheduler.listenerbus.eventqueue.capacity with this feature, according to #14269 (comment)

How was this patch tested?

a new test

[cloud-fan]

cc @JoshRosen@dhruve

[JoshRosen]

FWIW I didn't actually say that we should rename that key since the cost of the confusing name isn't that high right now. So while I don't oppose this mechanism I'm neutral on it given that the only use case so far seems kind of minor. I was mostly commenting just so that future readers and reviewers can more easily spot the issue and hopefully pick better names going forward.

…

On Thu, May 25, 2017 at 8:57 AM Apache Spark QA ***@***.***> wrote: *Test build #77372 has started <https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77372/testReport>* for PR 18110 at commit cc51dd0 <cc51dd0> . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#18110 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADGPMaYT_X0iqsDi6q5OhpOOxZpnKQzks5r9aT0gaJpZM4Nmjjt> .

[cloud-fan]

@JoshRosen actually there are more use cases. Currently we have 2 ways to define a config: create a config entry in the config package object, or SQLConf object, and read conf value with the config entry in code. Or read conf value with hard-coded conf name directly in the code. Ideally we should always create config entry, but we can't because it doesn't support alternative keys.

[JoshRosen]

@cloud-fan, what about SparkConf's configsWithAlternatives : https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkConf.scala#L596

[cloud-fan]

It's only used in the SparkCong.get(key: String) code path, not SparkConf.get(entry: ConfigEntry[T]) code path. That's why we only support alternative keys if users get conf value by hard-coded conf name.

[JoshRosen]

Ahhh, makes sense. Thanks for the clarification.

[SparkQA]

Test build #77372 has finished for PR 18110 at commit cc51dd0.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jiangxb1987]

LGTM only have a nit.

[jiangxb1987]

nit: add comment for this method.

[SparkQA]

Test build #77398 has finished for PR 18110 at commit 3f9d202.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jiangxb1987]

retest please

[SparkQA]

Test build #77411 has started for PR 18110 at commit a637e95.

[cloud-fan]

retest this please

[SparkQA]

Test build #77417 has finished for PR 18110 at commit a637e95.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[cloud-fan]

thanks for the review, merging to master!