[SPARK-36494] Add param bucketSpec when create LogicalRelation for the hive table in HiveMetastoreCatalog.covertToLogicalRelation

[zengruios]

What changes were proposed in this pull request?

Add param bucketSpec when create LogicalRelation for the hive table in HiveMetastoreCatalog.covertToLogicalRelation

Why are the changes needed?

If bucketSpec is not used, SortMergeJoin will do unnecessary shuffle.

Does this PR introduce any user-facing change?

Yes. See in SPARK-36494.

How was this patch tested?

Test it in my develop enviroinment.

[AmplabJenkins]

Can one of the admins verify this patch?

[HyukjinKwon]

@zengruios can we have a test?

[c21]

I don't think this PR works, as Hive bucketed table are using Hive hash, and is incompatible with Spark murmur3 hash. As discussed with @cloud-fan and @sunchao, we probably want to support Hive bucketed table read in data source v2 effort. btw Supporting for Hive bucketed table write is WIP - #33432 .

[pan3793]

we probably want to support Hive bucketed table read in data source v2 effort

@c21 What time does the community plan to migrate Hive to ds v2?

[github-actions]

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!