Uh oh!
There was an error while loading. Please reload this page.
[SPARK-10180] [SQL] JDBC datasource are not processing EqualNullSafe filter - #8743
[SPARK-10180] [SQL] JDBC datasource are not processing EqualNullSafe filter#8743HyukjinKwon wants to merge 9 commits into
Conversation
JoshRosen
commented
Sep 15, 2015
Please update the pull request description to describe the changes in this patch. |
JoshRosen
commented
Sep 15, 2015
Will this case be covered by the end-to-end docker tests? |
HyukjinKwon
commented
Sep 19, 2015
Should I better write some test codes for this separately? |
yhuai
commented
Nov 18, 2015
Can we add a test? |
HyukjinKwon
commented
Nov 20, 2015
I added a simple test for this. I wanted to add a test including |
HyukjinKwon
commented
Nov 20, 2015
test this please |
HyukjinKwon
commented
Nov 20, 2015
It looks Jenkins does not run the test for the past commits that I made as a user not added to whitelist. Would anybody run the test for this please if it looks good? |
yhuai
commented
Nov 20, 2015
test this please |
SparkQA
commented
Nov 20, 2015
Test build #46425 has finished for PR 8743 at commit
|
HyukjinKwon
commented
Dec 9, 2015
Just a question. Now it looks the PR for the end-to-end docker tests is merged. Do you think it needs all the tests for all the databases (namely |
rxin
commented
Dec 30, 2015
@zsxwing can you review this one? |
maropu
commented
Dec 30, 2015
@HyukjinKwon +1, I think it'd better to add tests in |
HyukjinKwon
commented
Dec 30, 2015
Hm.. actually I think adding tests more with the docker ingeration tests is a bit over-tested. It looks the comparison operators I used are all already being used in |
maropu
commented
Dec 30, 2015
Yes and you're right. We need to clearly define the rule that decides which tests should be added in |
There was a problem hiding this comment.
It took me several minutes to understand it. Maybe (attr IS NULL AND value IS NULL) OR (attr <> null AND value <> null AND attr = value) is easier.
There was a problem hiding this comment.
Ah, sorry, my previous expr doesn't work.
zsxwing
commented
Dec 31, 2015
I'm wondering if it's okey to add |
HyukjinKwon
commented
Dec 31, 2015
@zsxwing I will anyway resolve the conflicts first |
rxin
commented
Jan 1, 2016
Sorry I kept merging pull requests that made your life harder. Can you bring it up to date again? I promise this is the last one. |
Conflicts: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala
HyukjinKwon
commented
Jan 1, 2016
@rxin Nothing is easy and happy new year:) I just resolved conflicts. |
HyukjinKwon
commented
Jan 1, 2016
test this please |
SparkQA
commented
Jan 1, 2016
Test build #2287 has finished for PR 8743 at commit
|
HyukjinKwon
commented
Jan 1, 2016
test this please |
SparkQA
commented
Jan 2, 2016
Test build #2291 has finished for PR 8743 at commit
|
SparkQA
commented
Jan 2, 2016
Test build #2294 has finished for PR 8743 at commit
|
HyukjinKwon
commented
Jan 2, 2016
test this please |
SparkQA
commented
Jan 2, 2016
Test build #2295 has finished for PR 8743 at commit
|
rxin
commented
Jan 2, 2016
Thanks - I've merged this. |
This PR is followed by #8391.
Previous PR fixes JDBCRDD to support null-safe equality comparison for JDBC datasource. This PR fixes the problem that it can actually return null as a result of the comparison resulting error as using the value of that comparison.