Uh oh!
There was an error while loading. Please reload this page.
[SPARK-33503][SQL] Refactor SortOrder class to allow multiple childrens - #30430
[SPARK-33503][SQL] Refactor SortOrder class to allow multiple childrens#30430prakharjain09 wants to merge 10 commits into
Conversation
prakharjain09
commented
Nov 19, 2020
@cloud-fan@maropu I have made the changes as suggested in #30302 (comment) . |
HyukjinKwon
commented
Nov 20, 2020
ok to test |
maropu
commented
Nov 20, 2020
@prakharjain09 Thanks for working on this. btw, could you assign a new jira ID to this? |
| @@ -64,7 +64,9 @@ case class SortOrder( | |||
| direction: SortDirection, | |||
| nullOrdering: NullOrdering, | |||
| sameOrderExpressions: Set[Expression]) | |||
There was a problem hiding this comment.
We need to use Set for this variable?
There was a problem hiding this comment.
We can use ExpressionSet maybe at those 2 places and make it a Seq in constructor here. any thoughts?
Yea, I think it is okay to just deduplicate it before storing it in the class as you said, and then make it Seq.
Uh oh!
There was an error while loading. Please reload this page.
SparkQA
commented
Nov 20, 2020
Kubernetes integration test starting |
SparkQA
commented
Nov 20, 2020
Kubernetes integration test status failure |
SparkQA
commented
Nov 20, 2020
Test build #131386 has finished for PR 30430 at commit
|
SparkQA
commented
Nov 20, 2020
Kubernetes integration test starting |
SparkQA
commented
Nov 20, 2020
Kubernetes integration test status success |
SparkQA
commented
Nov 20, 2020
Test build #131426 has finished for PR 30430 at commit
|
Uh oh!
There was an error while loading. Please reload this page.
SparkQA
commented
Nov 25, 2020
Test build #131785 has finished for PR 30430 at commit
|
SparkQA
commented
Nov 27, 2020
Test build #131885 has finished for PR 30430 at commit
|
SparkQA
commented
Nov 27, 2020
Test build #131886 has finished for PR 30430 at commit
|
Uh oh!
There was an error while loading. Please reload this page.
The GA tests passed, so I merged to master. Thanks! |
What changes were proposed in this pull request?
This is a followup of #30302 . As part of this PR, sameOrderExpressions set is made part of children of SortOrder node - so that they don't need any special handling as done in #30302 .
Why are the changes needed?
sameOrderExpressions should get same treatment as child. So making them part of children helps in transforming them easily.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Existing UTs