[SPARK-35490][BUILD] Update json4s to 3.7.0-M11 by vinodkc · Pull Request #32636 · apache/spark

vinodkc · 2021-05-22T16:19:15Z

What changes were proposed in this pull request?

This PR aims to upgrade json4s from 3.7.0-M5 to 3.7.0-M11

Note: json4s version greater than 3.7.0-M11 is not binary compatible with Spark third party jars

Why are the changes needed?

Multiple defect fixes and improvements like

json4s/json4s#750
json4s/json4s#554
json4s/json4s#715

Does this PR introduce any user-facing change?

No

How was this patch tested?

Ran with the existing UTs

SparkQA · 2021-05-22T16:23:22Z

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43356/

SparkQA · 2021-05-22T16:27:34Z

Test build #138834 has finished for PR 32636 at commit 105ccae.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-05-23T02:28:25Z

Test build #138838 has finished for PR 32636 at commit ef6430b.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-05-23T02:34:12Z

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43360/

dongjoon-hyun · 2021-05-23T07:12:39Z

Hi, @vinodkc . Jenkins also failed with import error.

[error] /home/jenkins/workspace/SparkPullRequestBuilder/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryStatusAndProgressSuite.scala:24:19: Unused import
[error] import org.json4s._
[error] ^
[error] one error found
[error] (sql / Test / compileIncremental) Compilation failed
[error] Total time: 429 s (07:09), completed May 22, 2021 7:28:24 PM

Do we need to remove more imports?

SparkQA · 2021-05-23T10:21:56Z

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43363/

SparkQA · 2021-05-23T10:26:58Z

Test build #138841 has finished for PR 32636 at commit d38571f.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-05-23T11:27:24Z

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43366/

SparkQA · 2021-05-23T11:32:44Z

Test build #138844 has finished for PR 32636 at commit bbb2e0b.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-05-23T12:32:55Z

Test build #138846 has finished for PR 32636 at commit 20f4477.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-05-23T13:03:11Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43368/

SparkQA · 2021-05-23T13:34:10Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43368/

SparkQA · 2021-05-24T15:40:43Z

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43397/

SparkQA · 2021-05-24T15:44:47Z

Test build #138875 has finished for PR 32636 at commit 3026d9e.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-05-24T16:36:59Z

Test build #138876 has finished for PR 32636 at commit 7a8e65e.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

srowen · 2021-05-24T17:16:23Z

I think it's OK to ignore the MiMa warnings, as all affected methods are private.
The only slightly worrisome one is Row.jsonValue - it's private[sql] for sure but it's possible some third party libs try to extend Row or something. But hey, those are the risks if so.

SparkQA · 2021-05-24T17:28:04Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43398/

SparkQA · 2021-05-24T19:03:19Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43398/

SparkQA · 2021-05-25T07:16:17Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43430/

SparkQA · 2021-05-25T07:50:19Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43430/

SparkQA · 2021-05-25T09:08:11Z

Test build #138908 has finished for PR 32636 at commit 8cdbbb7.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-05-25T10:19:52Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43441/

SparkQA · 2021-05-25T10:55:28Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43441/

SparkQA · 2021-05-25T11:20:43Z

Test build #138920 has finished for PR 32636 at commit 8eeb868.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

srowen

This much seems OK then. I suppose eventually we'll have to deal with the binary change if it ends up in the final version.

dongjoon-hyun

+1, LGTM.

dongjoon-hyun · 2021-05-25T18:31:08Z

cc @MaxGekk , too.

MaxGekk

Multiple defect fixes and improvements

It would be nice to see some summary of the fixes/improvements. Maybe they are not relevant to Spark.

vinodkc · 2021-05-26T07:57:43Z

Multiple defect fixes and improvements
It would be nice to see some summary of the fixes/improvements. Maybe they are not relevant to Spark.

I've updated the summary of the fixes/improvements, thanks for the review comment

MaxGekk · 2021-05-26T08:09:37Z

+1, LGTM. Merging to master.
Thank you @vinodkc, and @srowen @dongjoon-hyun for your review.

github-actionsBot added the BUILD label May 22, 2021

vinodkc force-pushed the br_build_upgrade_json4s branch from 105ccae to ef6430bCompare May 23, 2021 02:16

github-actionsBot added the CORE label May 23, 2021

HyukjinKwon changed the title ~~[SPARK-35490][BUILD]Update json4s to 3.7.0-M16~~[SPARK-35490][BUILD] Update json4s to 3.7.0-M16May 23, 2021

dongjoon-hyun reviewed May 23, 2021
View reviewed changes

Comment threadcore/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala Outdated

vinodkc force-pushed the br_build_upgrade_json4s branch from ef6430b to d38571fCompare May 23, 2021 09:30

github-actionsBot added SQL STRUCTURED STREAMING WEB UI labels May 23, 2021

vinodkc force-pushed the br_build_upgrade_json4s branch from d38571f to bbb2e0bCompare May 23, 2021 11:22

vinodkc force-pushed the br_build_upgrade_json4s branch from bbb2e0b to 20f4477Compare May 23, 2021 11:43

vinodkc force-pushed the br_build_upgrade_json4s branch 2 times, most recently from 12483bd to 7a8e65eCompare May 24, 2021 16:04

vinodkc changed the title ~~[SPARK-35490][BUILD] Update json4s to 3.7.0-M16~~[SPARK-35490][BUILD] Update json4s to 3.7.0-M11May 25, 2021

Update json4s to 3.7.0-M11
8eeb868

vinodkc force-pushed the br_build_upgrade_json4s branch from 8cdbbb7 to 8eeb868Compare May 25, 2021 08:40

srowen approved these changes May 25, 2021
View reviewed changes

dongjoon-hyun approved these changes May 25, 2021
View reviewed changes

MaxGekk approved these changes May 25, 2021
View reviewed changes

MaxGekk closed this in e3c6907May 26, 2021

sarutak mentioned this pull request Aug 4, 2021
[SPARK-36408][BUILD] Upgrade json4s to 4.0.3 #33630
Closed

LuciferYang mentioned this pull request Apr 3, 2024
[SPARK-47706][BUILD] Bump json4s 4.0.7 #45838
Closed

Uh oh!

Conversation

vinodkc commented May 22, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

SparkQA commented May 22, 2021

Uh oh!

SparkQA commented May 22, 2021

Uh oh!

SparkQA commented May 23, 2021

Uh oh!

SparkQA commented May 23, 2021

Uh oh!

dongjoon-hyun commented May 23, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

SparkQA commented May 23, 2021

Uh oh!

SparkQA commented May 23, 2021

Uh oh!

SparkQA commented May 23, 2021

Uh oh!

SparkQA commented May 23, 2021

Uh oh!

SparkQA commented May 23, 2021

Uh oh!

SparkQA commented May 23, 2021

Uh oh!

SparkQA commented May 23, 2021

Uh oh!

SparkQA commented May 24, 2021

Uh oh!

SparkQA commented May 24, 2021

Uh oh!

SparkQA commented May 24, 2021

Uh oh!

srowen commented May 24, 2021

Uh oh!

SparkQA commented May 24, 2021

Uh oh!

SparkQA commented May 24, 2021

Uh oh!

SparkQA commented May 25, 2021

Uh oh!

SparkQA commented May 25, 2021

Uh oh!

SparkQA commented May 25, 2021

Uh oh!

SparkQA commented May 25, 2021

Uh oh!

SparkQA commented May 25, 2021

Uh oh!

SparkQA commented May 25, 2021

Uh oh!

srowen left a comment

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun left a comment

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun commented May 25, 2021

Uh oh!

MaxGekk left a comment

Choose a reason for hiding this comment

Uh oh!

vinodkc commented May 26, 2021

Uh oh!

MaxGekk commented May 26, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

vinodkc commented May 22, 2021•
edited
Loading

dongjoon-hyun commented May 23, 2021•
edited
Loading