Uh oh!
There was an error while loading. Please reload this page.
[SPARK-26466][CORE] Use ConfigEntry for hardcoded configs for submit categories. - #23532
[SPARK-26466][CORE] Use ConfigEntry for hardcoded configs for submit categories.#23532HeartSaVioR wants to merge 3 commits into
Conversation
SparkQA
commented
Jan 14, 2019
Test build #101151 has finished for PR 23532 at commit
|
HeartSaVioR
commented
Jan 14, 2019
Ran compilation in my machine and passed.
|
SparkQA
commented
Jan 14, 2019
Test build #101152 has finished for PR 23532 at commit
|
There was a problem hiding this comment.
I had to set this to false because by default it doesn't allow setting configuration which is defined in SparkConf. I'm not fully sure which one is right to do: 1) make the change as I just did 2) don't add spark.kryoserializer.buffer and spark.kryoserializer.buffer.max to Kryo.
SparkQA
commented
Jan 14, 2019
Test build #101174 has finished for PR 23532 at commit
|
srowen
left a comment
There was a problem hiding this comment.
This is looking good @HeartSaVioR ; could you rebase?
Following are addressed: * spark.kryo * spark.kryoserializer * spark.serializer * spark.jars * spark.files * spark.submit * spark.deploy * spark.worker
c263104 to
b00669dCompareHeartSaVioR
commented
Jan 16, 2019
@srowen Thanks for reviewing! I've just rebased. |
SparkQA
commented
Jan 16, 2019
Test build #101323 has finished for PR 23532 at commit
|
srowen
commented
Jan 17, 2019
Merged to master |
HeartSaVioR
commented
Jan 17, 2019
Thanks for reviewing and merging, @srowen ! |
…categories. ## What changes were proposed in this pull request? The PR makes hardcoded configs below to use `ConfigEntry`. * spark.kryo * spark.kryoserializer * spark.serializer * spark.jars * spark.files * spark.submit * spark.deploy * spark.worker This patch doesn't change configs which are not relevant to SparkConf (e.g. system properties). ## How was this patch tested? Existing tests. Closesapache#23532 from HeartSaVioR/SPARK-26466-v2. Authored-by: Jungtaek Lim (HeartSaVioR) <kabhwan@gmail.com> Signed-off-by: Sean Owen <sean.owen@databricks.com>
What changes were proposed in this pull request?
The PR makes hardcoded configs below to use
ConfigEntry.This patch doesn't change configs which are not relevant to SparkConf (e.g. system properties).
How was this patch tested?
Existing tests.