Uh oh!
There was an error while loading. Please reload this page.
[SPARK-26576][SQL][TEST] Broadcast hint not applied to partitioned table - #23530
[SPARK-26576][SQL][TEST] Broadcast hint not applied to partitioned table#23530jzhuge wants to merge 2 commits into
Conversation
## What changes were proposed in this pull request? Make sure broadcast hint is applied to partitioned tables. ## How was this patch tested? - A new unit test in PruneFileSourcePartitionsSuite - Unit test suites touched by SPARK-14581: JoinOptimizationSuite, FilterPushdownSuite, ColumnPruningSuite, and PruneFiltersSuite Closesapache#23507 from jzhuge/SPARK-26576. Authored-by: John Zhuge <jzhuge@apache.org> Signed-off-by: gatorsmile <gatorsmile@gmail.com> (cherry picked from commit b9eb0e8)
dongjoon-hyun
commented
Jan 13, 2019
ok to test |
| case h: ResolvedHint => | ||
| collectProjectsAndFilters(h.child) | ||
There was a problem hiding this comment.
Hi, @jzhuge .
Do we need to remove this? According to the previous PR, master has no issue. So, I expected a test only PR.
cc @cloud-fan and @gatorsmile
There was a problem hiding this comment.
Yeah, it is safer to only port the unit test. However, I believe this is dead code, otherwise we might have to revisit the fix for 2.4 and 2.3.
There was a problem hiding this comment.
In that case, let's not change this in this PR [SPARK-26576][SQL] Broadcast hint not applied to partitioned table .
For the dead code cleaning up, you can do that later.
SparkQA
commented
Jan 13, 2019
Test build #101135 has finished for PR 23530 at commit
|
SparkQA
commented
Jan 13, 2019
Test build #101139 has finished for PR 23530 at commit
|
gatorsmile
left a comment
There was a problem hiding this comment.
LGTM
Thanks! Merged to master.
## What changes were proposed in this pull request? Make sure broadcast hint is applied to partitioned tables. ## How was this patch tested? - A new unit test in PruneFileSourcePartitionsSuite - Unit test suites touched by SPARK-14581: JoinOptimizationSuite, FilterPushdownSuite, ColumnPruningSuite, and PruneFiltersSuite Closesapache#23507 from jzhuge/SPARK-26576. Closesapache#23530 from jzhuge/SPARK-26576-master. Authored-by: John Zhuge <jzhuge@apache.org> Signed-off-by: gatorsmile <gatorsmile@gmail.com>
What changes were proposed in this pull request?
Make sure broadcast hint is applied to partitioned tables.
How was this patch tested?
Closes#23507 from jzhuge/SPARK-26576.