Uh oh!
There was an error while loading. Please reload this page.
Bug fix in sendMessage() of pregel implementation in PageRank.scala - #20899
Bug fix in sendMessage() of pregel implementation in PageRank.scala#20899mangolzy wants to merge 1 commit into
Conversation
Iterator((edge.dstId, edge.srcAttr._2 * edge.attr)) -> Iterator((edge.dstId, edge.srcAttr._1 * edge.attr)) Since edge.srcAttr._2 is used to compare with tol, it should be the (newPR - oldPR), but in the sendMessage, the origin code send it as part of the message, instead it should be the newPR which is edge.srcAttr._1.
WenqianZhao
commented
Oct 31, 2018
WenqianZhao
commented
Oct 31, 2018
By the way, @ankurdave , hi, I'm very curious about why you use delta instead of just newPR as the message. Is that for better performance or just at chance? Could you please answer this question when you are free? Thank you so much. |
@WenqianZhao I think the point of sending deltas instead of absolute ranks was that, as parts of the graph converge, their deltas would go to zero. GraphX would then be able to compress those zero messages more efficiently. |
WenqianZhao
commented
Nov 1, 2018
Got it! Thank you! |
AmplabJenkins
commented
Apr 8, 2019
Can one of the admins verify this patch? |
What changes were proposed in this pull request?
Iterator((edge.dstId, edge.srcAttr._2 * edge.attr))
->
Iterator((edge.dstId, edge.srcAttr._1 * edge.attr))
How was this patch tested?
Since edge.srcAttr._2 is used to compare with tol, it should be the (newPR - oldPR), but in the sendMessage, the origin code send it as part of the message, instead it should be the newPR which is edge.srcAttr._1.
Please review http://spark.apache.org/contributing.html before opening a pull request.