Uh oh!
There was an error while loading. Please reload this page.
[SPARK-2375][SQL] JSON schema inference may not resolve type conflicts correctly for a field inside an array of structs - #1308
Closed
yhuai wants to merge 3 commits into
Closed
[SPARK-2375][SQL] JSON schema inference may not resolve type conflicts correctly for a field inside an array of structs#1308yhuai wants to merge 3 commits into
yhuai wants to merge 3 commits into
Conversation
AmplabJenkins
commented
Jul 5, 2014
Merged build triggered. |
AmplabJenkins
commented
Jul 5, 2014
Merged build started. |
Contributor
There was a problem hiding this comment.
Is this just a separate test you are adding? Seems like bad form to make these vars and reuse the variable name if not absolutely required.
AmplabJenkins
commented
Jul 5, 2014
Merged build triggered. |
AmplabJenkins
commented
Jul 5, 2014
Merged build started. |
AmplabJenkins
commented
Jul 5, 2014
Merged build finished. All automated tests passed. |
AmplabJenkins
commented
Jul 5, 2014
All automated tests passed. |
AmplabJenkins
commented
Jul 5, 2014
Merged build finished. All automated tests passed. |
AmplabJenkins
commented
Jul 5, 2014
All automated tests passed. |
asfgit pushed a commit
that referenced
this pull request
Jul 8, 2014
…s correctly for a field inside an array of structs
For example, for
```
{"array": [{"field":214748364700}, {"field":1}]}
```
the type of field is resolved as IntType. While, for
```
{"array": [{"field":1}, {"field":214748364700}]}
```
the type of field is resolved as LongType.
JIRA: https://issues.apache.org/jira/browse/SPARK-2375
Author: Yin Huai <huaiyin.thu@gmail.com>
Closes#1308 from yhuai/SPARK-2375 and squashes the following commits:
3e2e312 [Yin Huai] Update unit test.
1b2ff9f [Yin Huai] Merge remote-tracking branch 'upstream/master' into SPARK-237510794eb [Yin Huai] Correctly resolve the type of a field inside an array of structs.
(cherry picked from commit f0496ee)
Signed-off-by: Michael Armbrust <michael@databricks.com>marmbrus
commented
Jul 8, 2014
Contributor
Thanks, merged into master and 1.0 |
xiliu82 pushed a commit
to xiliu82/spark
that referenced
this pull request
Sep 4, 2014
…s correctly for a field inside an array of structs
For example, for
```
{"array": [{"field":214748364700}, {"field":1}]}
```
the type of field is resolved as IntType. While, for
```
{"array": [{"field":1}, {"field":214748364700}]}
```
the type of field is resolved as LongType.
JIRA: https://issues.apache.org/jira/browse/SPARK-2375
Author: Yin Huai <huaiyin.thu@gmail.com>
Closesapache#1308 from yhuai/SPARK-2375 and squashes the following commits:
3e2e312 [Yin Huai] Update unit test.
1b2ff9f [Yin Huai] Merge remote-tracking branch 'upstream/master' into SPARK-2375
10794eb [Yin Huai] Correctly resolve the type of a field inside an array of structs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For example, for
the type of field is resolved as IntType. While, for
the type of field is resolved as LongType.
JIRA: https://issues.apache.org/jira/browse/SPARK-2375