Uh oh!
There was an error while loading. Please reload this page.
[SPARK-21600][docs] The description of "this requires spark.shuffle.service.enabled to be set" for the spark.dynamicAllocation.enabled configuration item is not clear - #18806
Conversation
….enabled to be set" for the spark.dynamicAllocation.enabled configuration item is not clear. I am not sure how to set spark.shuffle.service.enabled is true or false, so that the user to guess, resulting in doubts. All i have changed here, stressed that must spark.shuffle.service.enabled to be set true.
AmplabJenkins
commented
Aug 2, 2017
Can one of the admins verify this patch? |
| <a href="job-scheduling.html#dynamic-resource-allocation">here</a>. | ||
| <br><br> | ||
| This requires <code>spark.shuffle.service.enabled</code> to be set. | ||
| This requires <code>spark.shuffle.service.enabled</code> to be set true. |
There was a problem hiding this comment.
I think there's no ambiguity here. Usually configuration with name "xxx.enabled" can only have two values "true" or "false". So "to be set" usually means to enable it (to set it to true).
There was a problem hiding this comment.
You are right, but usually is not very sure. Can not let the user to guess, the document needs to be accurately described. In addition, the spark project, several other are clearly described, spark.shuffle.service.enabled set true.
There was a problem hiding this comment.
Since other places are clearly defined the property, so there should be no ambiguity. Personally I'm not fond of this super nit fix...
There was a problem hiding this comment.
Thank you for your comments.
This requires spark.shuffle.service.enabled to be set true. It is very clearly. Only such an accurate description，there be no ambiguity.
guoxiaolongzte
commented
Aug 2, 2017
@srowen Help review the code,Thanks. |
guoxiaolongzte
commented
Aug 11, 2017
@srowen Help review the code,Thanks. |
srowen
commented
Aug 11, 2017
"set" is pretty synonymous with "true" for boolean properties. Its name includes 'enabled'. I think this is too trivial. |
Closesapache#18522Closesapache#17722Closesapache#18879Closesapache#18891Closesapache#18806Closesapache#18948Closesapache#18949Closesapache#19070Closesapache#19039Closesapache#19142Closesapache#18515Closesapache#19154Closesapache#19162Closesapache#19187
Closesapache#18522Closesapache#17722Closesapache#18879Closesapache#18891Closesapache#18806Closesapache#18948Closesapache#18949Closesapache#19070Closesapache#19039Closesapache#19142Closesapache#18515Closesapache#19154Closesapache#19162Closesapache#19187Closesapache#19091 Author: Sean Owen <sowen@cloudera.com> Closesapache#19203 from srowen/CloseStalePRs3.
What changes were proposed in this pull request?
The description of "this requires spark.shuffle.service.enabled to be set" for the spark.dynamicAllocation.enabled configuration item is not clear. I am not sure how to set spark.shuffle.service.enabled is true or false, so that the user to guess, resulting in doubts. All i have changed here, stressed that must spark.shuffle.service.enabled to be set true.
When I set spark.dynamicAllocation.enabled=true, but set spark.shuffle.service.enabled false, When I submit the spark-submit --master yarn job, the program throws the exception.
How was this patch tested?
manual tests
Please review http://spark.apache.org/contributing.html before opening a pull request.