Uh oh!
There was an error while loading. Please reload this page.
[MINOR] Remove extra anonymous closure within functional transformations - #12382
[MINOR] Remove extra anonymous closure within functional transformations#12382HyukjinKwon wants to merge 1 commit into
Conversation
HyukjinKwon
commented
Apr 14, 2016
cc @rxin |
rxin
commented
Apr 14, 2016
LGTM pending tests. |
SparkQA
commented
Apr 14, 2016
Test build #55793 has finished for PR 12382 at commit
|
HyukjinKwon
commented
Apr 14, 2016
retest this please |
| val words = lines.flatMap(_.split(" ")) | ||
| val wordCounts = words.map(x => (x, 1)).reduceByKey(_ + _) | ||
| wordCounts.foreachRDD((rdd: RDD[(String, Int)], time: Time) => { | ||
| wordCounts.foreachRDD { (rdd: RDD[(String, Int)], time: Time) => |
There was a problem hiding this comment.
While we're here, can this just be (rdd, time) =>? and likewise in the following file, maybe others
SparkQA
commented
Apr 14, 2016
Test build #55802 has finished for PR 12382 at commit
|
srowen
commented
Apr 14, 2016
Merged to master |
andrewor14
commented
Apr 14, 2016
Maybe we should add a scalastyle rule for this |
@andrewor14 Hm.. I tried to add a rule but I realised (or I think) it cannot find perfectly by a static way. One of the problems I met was below: For example, there is a class graph.aggregateMessages(ctx => { ctx.sendToSrc(1); ctx.sendToDst(1) }, _ + _,
TripletFields.None)
}In this case, this should not be the error case since In order to check this, it should be able to check Maybe I am not smart enough but this was just my conclusion. |
@andrewor14@srowen Do you mind If I open another (minor) PR for some more corrections? While trying to test regular expressions, I found some more things to correct about this and about @srowen comment, which are pretty many. +If you think it is able to add a rule, then I won't. |
srowen
commented
Apr 15, 2016
If there's a built-in scalastyle rule for this, let's enable it, but otherwise not sure it's worth building custom code to enforce it. If you've found ways to look for more instances of this and found more classes of this problem, OK open a new PR. |
…riting apache#14323 [MINOR] Remove extra anonymous closure within functional transformations apache#12382 just JdbcUtils.scala
What changes were proposed in this pull request?
This PR removes extra anonymous closure within functional transformations.
For example,
.map(item => { ... })which can be just simply as below:
.map { item => ... }How was this patch tested?
Related unit tests and
sbt scalastyle.