Uh oh!
There was an error while loading. Please reload this page.
[SPARK-11946][SQL] Audit pivot API for 1.6. - #9929
Conversation
rxin
commented
Nov 24, 2015
cc @aray |
There was a problem hiding this comment.
@aray do you know why we have a "sort" in here?
There was a problem hiding this comment.
The sort is there to ensure that the output columns are in a consistent logical order.
There was a problem hiding this comment.
ok thanks - i'm going to add a comment there to explain.
There was a problem hiding this comment.
this is done as part of #9603 (comment) but it is way too small to deserve its own pr.
SparkQA
commented
Nov 24, 2015
Test build #46591 has finished for PR 9929 at commit
|
There was a problem hiding this comment.
Should we use _to_java_column(pivot_col) and _to_seq() here? or df.pivot(df.a) may fail
SparkQA
commented
Nov 24, 2015
Test build #2104 has finished for PR 9929 at commit
|
davies
commented
Nov 24, 2015
LGTM |
rxin
commented
Nov 24, 2015
Thanks - merging this in. |
Currently pivot's signature looks like ```scala scala.annotation.varargs def pivot(pivotColumn: Column, values: Column*): GroupedData scala.annotation.varargs def pivot(pivotColumn: String, values: Any*): GroupedData ``` I think we can remove the one that takes "Column" types, since callers should always be passing in literals. It'd also be more clear if the values are not varargs, but rather Seq or java.util.List. I also made similar changes for Python. Author: Reynold Xin <rxin@databricks.com> Closes#9929 from rxin/SPARK-11946. (cherry picked from commit f315272) Signed-off-by: Reynold Xin <rxin@databricks.com>
Currently pivot's signature looks like
I think we can remove the one that takes "Column" types, since callers should always be passing in literals. It'd also be more clear if the values are not varargs, but rather Seq or java.util.List.
I also made similar changes for Python.