[SPARK-29483][BUILD] Bump Jackson to 2.10.0 by Fokko · Pull Request #26131 · apache/spark

Fokko · 2019-10-15T18:07:09Z

What changes were proposed in this pull request?

Release blog: https://medium.com/@cowtowncoder/jackson-2-10-features-cd880674d8a2

Fixes the following CVE's:
https://www.cvedetails.com/cve/CVE-2019-16942/
https://www.cvedetails.com/cve/CVE-2019-16943/

Looking back, there were 3 major goals for this minor release:

Resolve the growing problem of “endless CVE patches”, a stream of fixes for reported CVEs related to “Polymorphic Deserialization” problem (described in “On Jackson CVEs… ”) that resulted in security tools forcing Jackson upgrades. 2.10 now includes “Safe Default Typing” that is hoped to resolve this problem.
Evolve 2.x API towards 3.0, based on changes that were done in master, within limits of 2.x API backwards-compatibility requirements.
Add JDK support for versions beyond Java 8: specifically add“module-info.class” for JDK9+, defining proper module definitions for Jackson components

Full changelog: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.10

Improved Scala 2.13 support: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.10#scala

Why are the changes needed?

Patches CVE's reported by the vulnerability scanner.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Ran mvn clean install -DskipTests locally.

dongjoon-hyun · 2019-10-15T18:22:21Z

ok to test

dongjoon-hyun · 2019-10-15T18:23:04Z

Please polish the PR description, @Fokko .

Also, cc @srowen since this is Scala 2.13-related.

dongjoon-hyun · 2019-10-15T18:24:49Z

BTW, @Fokko . We use [BUILD] instead of [CORE] for this kind of PRs~

Fixes the following CVE's: https://www.cvedetails.com/cve/CVE-2019-16942/https://www.cvedetails.com/cve/CVE-2019-16943/

srowen · 2019-10-15T20:33:48Z

+jackson-module-jaxb-annotations-2.10.0.jar
+jackson-module-paranamer-2.10.0.jar
+jackson-module-scala_2.12-2.10.0.jar
+jakarta.activation-api-1.2.1.jar


We'll need an entry in LICENSE-binary for this, under the Eclipse Dist License section. (We seem to already have a copy of the license though in licenses-binary).

Done, thanks for pointing out

SparkQA · 2019-10-15T21:01:46Z

Test build #112125 has finished for PR 26131 at commit 7314e1e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

srowen

Looks OK pending tests.

SparkQA · 2019-10-16T10:27:46Z

Test build #112152 has finished for PR 26131 at commit 43e3d5c.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-10-16T19:20:38Z

Test build #4898 has finished for PR 26131 at commit 43e3d5c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun

+1, LGTM. Thank you, @Fokko and @srowen .
Merged to master.

dongjoon-hyun · 2019-10-16T22:39:31Z

Could you make a back porting PR, @Fokko ?

dongjoon-hyun · 2019-10-16T22:43:51Z

cc @jiangxb1987 since he is the release manager for 3.0.0-preview.

srowen · 2019-10-16T22:51:38Z

Not a back port to 2.4 right? We can't update Jackson in 2.4. I don't think we have to back port to the preview branch, even if we keep it.

dongjoon-hyun · 2019-10-16T23:06:20Z

Oh, I see. Got it, @srowen .

krishna-pandey · 2019-10-18T11:06:07Z

@srowen What's the issue in upgrading Jackson in 2.4?

Fokko · 2019-10-18T12:00:49Z

@krishna-pandey please refer to #21596

Jackson is notorious for having to change their public API, and aggressively deprecating methods. This will conflict with older versions of Hadoop. This might have huge implications for your application.

Release blog: https://medium.com/cowtowncoder/jackson-2-10-features-cd880674d8a2 Fixes the following CVE's: https://www.cvedetails.com/cve/CVE-2019-16942/https://www.cvedetails.com/cve/CVE-2019-16943/ Looking back, there were 3 major goals for this minor release: - Resolve the growing problem of “endless CVE patches”, a stream of fixes for reported CVEs related to “Polymorphic Deserialization” problem (described in “On Jackson CVEs… ”) that resulted in security tools forcing Jackson upgrades. 2.10 now includes “Safe Default Typing” that is hoped to resolve this problem. - Evolve 2.x API towards 3.0, based on changes that were done in master, within limits of 2.x API backwards-compatibility requirements. - Add JDK support for versions beyond Java 8: specifically add“module-info.class” for JDK9+, defining proper module definitions for Jackson components Full changelog: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.10 Improved Scala 2.13 support: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.10#scala Patches CVE's reported by the vulnerability scanner. No Ran `mvn clean install -DskipTests` locally. Closesapache#26131 from Fokko/SPARK-29483. Authored-by: Fokko Driesprong <fokko@apache.org> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>

Fokko changed the title ~~[SPARK-29483][BUILD] Bump Jackson to 2.10.0~~[SPARK-29483][CORE] Bump Jackson to 2.10.0Oct 15, 2019

Fokko force-pushed the SPARK-29483 branch from 8a0e5d8 to f9703b3Compare October 15, 2019 18:09

dongjoon-hyun changed the title ~~[SPARK-29483][CORE] Bump Jackson to 2.10.0~~[SPARK-29483][BUILD] Bump Jackson to 2.10.0Oct 15, 2019

[SPARK-29483][BUILD] Bump Jackson to 2.10.0
7314e1e
Fixes the following CVE's: https://www.cvedetails.com/cve/CVE-2019-16942/https://www.cvedetails.com/cve/CVE-2019-16943/

Fokko force-pushed the SPARK-29483 branch from f9703b3 to 7314e1eCompare October 15, 2019 18:25

dongjoon-hyun added the BUILD label Oct 15, 2019

srowen requested changes Oct 15, 2019
View reviewed changes

Add jakarta.activation:jakarta.activation-api in LICENSE-binary
43e3d5c

srowen approved these changes Oct 16, 2019
View reviewed changes

dongjoon-hyun approved these changes Oct 16, 2019
View reviewed changes

dongjoon-hyun closed this in 8eb8f74Oct 16, 2019

Fokko deleted the SPARK-29483 branch October 18, 2019 12:00

mccheah mentioned this pull request Nov 26, 2019
[SPARK-29483][BUILD] Bump Jackson to 2.10.0 palantir/spark#627
Merged

MaxGekk mentioned this pull request Dec 8, 2019
[SPARK-30166][SQL] Eliminate compilation warnings in JSONOptions #26797
Closed

Uh oh!

Conversation

Fokko commented Oct 15, 2019• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

dongjoon-hyun commented Oct 15, 2019

Uh oh!

dongjoon-hyun commented Oct 15, 2019

Uh oh!

dongjoon-hyun commented Oct 15, 2019

Uh oh!

srowenOct 15, 2019

Choose a reason for hiding this comment

Uh oh!

FokkoOct 16, 2019

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Oct 15, 2019

Uh oh!

srowen left a comment

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Oct 16, 2019

Uh oh!

SparkQA commented Oct 16, 2019

Uh oh!

dongjoon-hyun left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun commented Oct 16, 2019

Uh oh!

dongjoon-hyun commented Oct 16, 2019

Uh oh!

srowen commented Oct 16, 2019

Uh oh!

dongjoon-hyun commented Oct 16, 2019

Uh oh!

krishna-pandey commented Oct 18, 2019

Uh oh!

Fokko commented Oct 18, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Fokko commented Oct 15, 2019•
edited
Loading

dongjoon-hyun left a comment •
edited
Loading