Skip to content

Add msgpack-jackson3 module for Jackson 3.x support - #987

Open
komamitsu wants to merge 4 commits into
msgpack:mainfrom
komamitsu:support-jackson3
Open

Add msgpack-jackson3 module for Jackson 3.x support#987
komamitsu wants to merge 4 commits into
msgpack:mainfrom
komamitsu:support-jackson3

Conversation

@komamitsu

@komamitsukomamitsu commented May 31, 2026

Copy link
Copy Markdown
Member

Summary

  • Adds a new msgpack-jackson3 submodule (jackson-dataformat-msgpack3) that ports the existing Jackson 2.x integration to Jackson 3.x (tools.jackson namespace)
  • Requires Java 17+ and Jackson 3.1.2; the module is excluded from the build on older JDKs so existing Java 8 users are unaffected
  • Includes JMH benchmarks (src/jmh/java) runnable via ./sbt "msgpack-jackson3/jmh:run"
  • FixesAdd Jackson 3 support #933

Key differences from msgpack-jackson

  • Package namespace: com.fasterxml.jacksontools.jackson
  • TokenStreamLocation replaces JsonLocation; byte offset passed as columnNr
  • Lightweight MessagePackWriteContext replaces SimpleStreamWriteContext

CopilotAI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new msgpack-jackson3 submodule that ports the existing Jackson 2.x MessagePack integration to Jackson 3.x (tools.jackson namespace). It is published as a separate artifact (jackson-dataformat-msgpack-jackson3), requires Java 17+, and is conditionally included in the SBT build / CI / release pipelines so existing Java 8 users of msgpack-jackson are unaffected.

Changes:

  • New msgpack-jackson3 module mirroring msgpack-jackson, adapted for Jackson 3 APIs (ObjectWriteContext/ObjectReadContext, TokenStreamLocation, ValueSerializer/ValueDeserializer, builder-style MapperBuilder/TSFBuilder, etc.) and including a lightweight MessagePackWriteContext replacement for SimpleStreamWriteContext.
  • SBT build, CI workflow, and release workflow updated to compile/test/publish the new module only on JDK 17+, plus a new sbt-jmh plugin and JMH benchmarks under src/jmh/java.
  • Numerous regression tests covering tricky edge cases (null keys, write-context state, dup-detection NPEs, root-scalar ordering, multi-root containers, writeNumber(String) NaN/Infinity, etc.).

Reviewed changes

Copilot reviewed 41 out of 42 changed files in this pull request and generated no comments.

Show a summary per file
FileDescription
build.sbtAdds msgpack-jackson3 project, JMH plugin, Java 17 gating via isJava17Plus.
project/plugins.sbtAdds sbt-jmh plugin.
.github/workflows/CI.ymlSkips msgpack-jackson3 tests on JDKs < 17.
.github/workflows/release.ymlSplits publishing so jackson3 is only published with JDK 17.
msgpack-jackson3/README.mdNew documentation for the Jackson 3 module.
msgpack-jackson3/src/main/java/.../MessagePackFactory.javaJackson 3 BinaryTSFactory implementation.
msgpack-jackson3/src/main/java/.../MessagePackFactoryBuilder.javaNew TSFBuilder for the factory.
msgpack-jackson3/src/main/java/.../MessagePackGenerator.javaPort of generator; uses new MessagePackWriteContext, node-buffering model.
msgpack-jackson3/src/main/java/.../MessagePackParser.javaPort of parser to ParserMinimalBase; uses MessagePackReadContext.
msgpack-jackson3/src/main/java/.../MessagePackReadContext.javaRead context (replaces JsonReadContext).
msgpack-jackson3/src/main/java/.../MessagePackWriteContext.javaLightweight write context.
msgpack-jackson3/src/main/java/.../MessagePackMapper.javaObjectMapper subclass with new builder API.
msgpack-jackson3/src/main/java/.../MessagePackSerializerFactory.javaCustom serializer factory installing key serializer.
msgpack-jackson3/src/main/java/.../MessagePackSerializedString.javaSerializableString impl tolerant of null values.
msgpack-jackson3/src/main/java/.../MessagePackKeySerializer.javaKey serializer using MessagePackSerializedString.
msgpack-jackson3/src/main/java/.../MessagePackExtensionType.javaExtension type value + serializer.
msgpack-jackson3/src/main/java/.../ExtensionTypeCustomDeserializers.javaPer-type extension deserializer registry.
msgpack-jackson3/src/main/java/.../TimestampExtensionModule.javaModule serializing Instant as msgpack timestamp ext type.
msgpack-jackson3/src/main/java/.../JsonArrayFormat.javaAnnotationIntrospector forcing ARRAY shape.
msgpack-jackson3/src/main/java/.../PackageVersion.javaHard-coded package version.
msgpack-jackson3/src/main/java/.../Tuple.javaSmall Tuple helper.
msgpack-jackson3/src/test/java/...Ported test suite plus new regression tests for Jackson 3 specific behavior.
msgpack-jackson3/src/jmh/java/...New JMH benchmarks (read/write/UTF-8) and model classes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@komamitsu
komamitsu requested a review from xerialMay 31, 2026 14:27
@komamitsu

Copy link
Copy Markdown
MemberAuthor

@xerial Can you take a quick look at the changes of .github/workflows/* and *.sbt?

@xerial

Copy link
Copy Markdown
Member

@komamitsu Do you still need to maintain msgpack-jackson for Jackson 2.x? If we don't have critical use cases, I'll lean to dropping Jackson 2 support as supporting both Jackson2 and 3 will be overkill for msgpack-java project.

@xerial

Copy link
Copy Markdown
Member

@komamitsu

Copy link
Copy Markdown
MemberAuthor

@xerial Jackson 2 is still actively used, and Jackson 3 requires Java 17+, so not everyone can migrate yet. I think we should support both for a while.

Address PR msgpack#987 feedback asking for a clear artifact-name mapping
between jackson-dataformat-msgpack (Jackson 2) and
jackson-dataformat-msgpack-jackson3 (Jackson 3) to reduce upgrade
confusion.
main migrated the build to sbt 2 (feb87ef) without sbt-jmh, since
sbt-jmh 0.4.7 has no sbt-2-compatible artifact. This branch's
msgpack-jackson3 JMH benchmarks depend on sbt-jmh, so after merging
main the plugin fails to resolve. 0.4.8 publishes an sbt_2_3 build.
@komamitsu

Copy link
Copy Markdown
MemberAuthor

@hiSandog Good point. Addressed in 531d3d0

@komamitsu

Copy link
Copy Markdown
MemberAuthor

@xerial I addressed the conflict, PTAL.

BTW, it seems Universal Buffer Test has been skipped since #998 was merged.

@xerial

Copy link
Copy Markdown
Member

@komamitsu Thank you for this PR — the Jackson 3 port itself looks solid, and I agree with your point that we should support both Jackson 2 and 3 for a while, since Jackson 3 requires Java 17+ and many users can't migrate yet.

Before merging, I'd like to propose restructuring the module naming. My concern with jackson-dataformat-msgpack3 is long-term: if Jackson 3 takes over Jackson 2 in the next couple of years, the unmarked name jackson-dataformat-msgpack will be stuck on the legacy version, and Jackson 3 users (the growing cohort) would eventually need a second migration from -msgpack3 back to the plain name. I'd rather put the clean name on the future from day one.

Concretely, I propose we cut msgpack-java 1.0.0 with this structure:

  • jackson-dataformat-msgpack (1.x) = Jackson 3: your Jackson 3 code from this PR becomes the contents of the msgpack-jackson module. To avoid a classpath clash with the Jackson 2 artifact, the Java package moves to org.msgpack.jackson3.dataformat. Since Jackson 3 adopters are already rewriting every com.fasterxml.jackson import to tools.jackson, one more changed import costs them almost nothing — and it means both artifacts can coexist in one application for incremental migration (which was a design goal of Jackson's own tools.jackson rename).
  • jackson2-dataformat-msgpack (1.x) = Jackson 2: the current msgpack-jackson code moves here unchanged, keeping the org.msgpack.jackson.dataformat package. Jackson 2 users who want to keep receiving msgpack-core updates only need a one-line artifactId change — no import changes at all. This module would be maintenance-mode (Jackson 2.x bumps and bug fixes only), documented as such in the README, to be dropped in a future major release when Jackson 2 usage fades.
  • 0.9.x freezes: users who do nothing stay on jackson-dataformat-msgpack 0.9.10 and nothing breaks. We'll keep a 0.9.x branch as an escape hatch for critical fixes, but the plan is zero releases from it — this avoids maintaining two release lines with msgpack-core backports.

If dependabot/renovate proposes 0.9.x → 1.0.0 to a Jackson 2 user, it fails loudly at compile time (the old package is gone from that artifact), and the release notes will point to jackson2-dataformat-msgpack — so there's no silent breakage.

Since msgpack-java's API has been stable for a decade while sitting at 0.x, I think this transition is also the right occasion to finally signal 1.0.0.

Mechanically this reuses essentially all of your work — the Jackson 3 sources just land in msgpack-jackson with the package rename, and the JDK-conditional build exclusion you added carries over (the module still requires Java 17+ while msgpack-core and the Jackson 2 module stay on Java 8). The OSGi bundle symbolic names and export packages would be updated to match.

WDYT? If you're on board, I'm happy to help with the build.sbt / workflow changes for the restructuring.

@xerial

Copy link
Copy Markdown
Member

I've opened #1012 implementing the restructuring proposed above, built on top of this PR's commits (so all the Jackson 3 port work is preserved and attributed). @komamitsu PTAL — if you prefer this direction we can continue the review there; if not, happy to discuss further here.

While enabling the module restructuring I also found that the Jackson 2 module's JUnit 5 tests have been silently skipped on main (no Jupiter test interface in the sbt build — likely related to the skipped Universal Buffer Test you noticed), and fixing that surfaced a small JsonLocation regression from #903. Both are fixed in #1012.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Jackson 3 support

3 participants

@komamitsu@xerial