Uh oh!
There was an error while loading. Please reload this page.
[SPARK-40136][SQL] Fix the fragment of SQL query contexts - #37566
[SPARK-40136][SQL] Fix the fragment of SQL query contexts#37566MaxGekk wants to merge 4 commits into
Conversation
MaxGekk
commented
Aug 18, 2022
@cloud-fan@HyukjinKwon Could you review this bug fix, please. |
| class SQLQueryContextSuite extends SparkFunSuite with SQLHelper { | ||
| test("SPARK-40136: the length of the fragment in query context") { |
There was a problem hiding this comment.
this basically tests String.substring... can we have an end-to-end test?
There was a problem hiding this comment.
@cloud-fan The failed test DIVIDE_BY_ZERO: can't divide an integer by zero *** FAILED *** is an end-to-end test actually. Let me fix it.
There was a problem hiding this comment.
Great! Can we remove this test suite then? The code we are testing is simply sqlText.get.substring(originStartIndex.get, originStopIndex.get + 1)
There was a problem hiding this comment.
I wouldn't remove this unit tests because:
- Current implementation of
fragmentis not onlysubstring(). It is a combination of isValid + substring. - The end-to-end tests don't check corner cases.
SQLQueryContext has poor test coverage currently. Instead of to hope that end-to-end tests guarantee its correctness, I would add more simple unit tests to SQLQueryContext.
MaxGekk
commented
Aug 18, 2022
The failure of |
MaxGekk
commented
Aug 18, 2022
Merging to master. Thank you, @cloud-fan@dongjoon-hyun for review. |
What changes were proposed in this pull request?
In the PR, I propose to change implementation of
fragmentinSQLQueryContextto get the fragment correctly. Before the changes,fragmentmisses the last char becauseoriginStopIndex.getpoints to the last char of fragments butsubstring()gets a sub-string up toendIndex - 1.Why are the changes needed?
It fixes a bug.
Does this PR introduce any user-facing change?
Yes.
How was this patch tested?
By running new test suite: