[SPARK-3698][SQL] Correctly check case sensitivity in GetField

[cloud-fan]

This PR is a follow up to #2382
It fix a bug when resolve something like a.b[0].c.d, #2382 only do case sensitive check when resolve Unresolved("a.b") to GetField(Attribute("a"), "b"), but not c and d.

[AmplabJenkins]

Can one of the admins verify this patch?

[cloud-fan]

For something like a.b[0].c.d, the origin logic here only works for a and b. but not c and d. So I just simplified the logic here and let the ResolveGetField rule to do its job.

[liancheng]

Would you mind to file a JIRA ticket for this PR?

[SparkQA]

QA tests have started for PR 2543 at commit 5b0a2d0.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2543 at commit 5b0a2d0.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[marmbrus]

If possible, I think it might be clearer to keep the resolver logic in the Analyzer rule.

[cloud-fan]

I was going to put this logic into Analyzer rule, but found some tests depend on GetField(child, fieldName), so I have to create this constructor of GetField. And these two are so similar, so I combine them together. Maybe I should fix those tests instead?

[marmbrus]

Thanks for working on this! A few minor comments.

[cloud-fan]

Hi @marmbrus , I have updated my PR according to your comments. Do you mind review it again?

[SparkQA]

QA tests have started for PR 2543 at commit c5b9106.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2543 at commit c5b9106.

• This patch fails unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • println(s"Failed to load main class $childMainClass.")
  • case class GetField(child: Expression, fieldName: String, findField: StructField => Boolean = null)

[marmbrus]

We don't need findField anymore do we?

[cloud-fan]

@marmbrus As commented before, I think we should handle GetItem and GetField differently even they are very similar. For GetItem, we need calculate the ordinal by evaluating an expression, for GetField, we can calculate the ordinal by searching the corrected StrcutField according to fieldName which I think should be done in analyze phase. It may be better to put StructFiled into GetFiled but not a simple fieldName.

[cloud-fan]

Ping @marmbrus@liancheng I have finished the code locally, if you vote for UnresolvedGetField, I can push the code immediately.

[cloud-fan]

Hi @marmbrus@liancheng, I think it's better to calculate the ordinal of GetField in analyze phase, and I have updated the code to introduce the UnresolvedGetField, and fix the case-sensitivity-check bug.
Please let me know if you have any questions. Thanks!

[marmbrus]

Sorry for the delay merging this, but I have been concerned that we are adding unnecessary complexity to analysis by adding more types of expressions. I've built a simpler solution in #3724 based off the test case that you provided. If that looks reasonable to you I suggest we close this issue.