Decode JWS payloads as UTF-8 instead of the JVM default charset

[matteobaccan]

The problem

SignedDataVerifier.parseJWTPayload converts the base64url-decoded payload bytes into a String without specifying a charset:

String payload = new String(Base64.getUrlDecoder().decode(jwt.getPayload()));

new String(byte[]) uses the JVM default charset. A JWT Claims Set is always a UTF-8 encoded JSON object (RFC 7519 §3, RFC 8259 §8.1), so the two only agree when the JVM happens to run with UTF-8 as its default.

That is not guaranteed on the Java versions this library supports. JEP 400 made UTF-8 the default only in Java 18; on Java 11 through 17 the default charset comes from the platform locale, so a server running on Windows or with a non-UTF-8 LANG gets windows-1252, ISO-8859-1, Shift_JIS, and so on.

When that happens, every non-ASCII character in a decoded payload is corrupted silently. The signature is verified against the raw JWS, so verification still succeeds and no exception is raised — the caller simply receives mojibake. This affects every entry point that goes through decodeSignedObject: verifyAndDecodeTransaction, verifyAndDecodeRenewalInfo, verifyAndDecodeNotification, verifyAndDecodeAppTransaction and verifyAndDecodeRealtimeRequest.

It is easy to reach in practice. The Advanced Commerce descriptors and items carry merchant-supplied free text (displayName, description), so any app that is not English-only can hit it.

Reproduction

On JDK 17 with windows-1252 as the default charset, decoding a signed transaction whose advancedCommerceInfo.descriptors.description is Abonnement Café — 5,99 € par mois returns:

Abonnement Café — 5,99 € par mois

The existing test suite cannot catch this: all of its payload fixtures are ASCII-only, and CI sets JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, which hides the difference.

The fix

new String(bytes, StandardCharsets.UTF_8) — one argument.

Source encoding

The build does not set options.encoding, so javac reads the sources with the platform charset too. 25 files under src/main contain non-ASCII characters (typographic apostrophes in the javadoc), which means those are corrupted in locally built javadoc on a non-UTF-8 machine. It also makes a regression test for this bug impossible to write in the natural way: the expected string literal would be corrupted by the compiler in exactly the same way as the actual value, and the assertion would pass.

This PR sets options.encoding = 'UTF-8' on the JavaCompile tasks and on javadoc. Note that ci-prb.yml currently compensates for the missing setting with JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, while ci-release-javadocs.yml does not set it at all.

The test

SignedDataVerifierTest.testNonAsciiDataDecodingIsIndependentOfTheDefaultCharset decodes a new fixture containing French and Japanese text. Its expected values are written as \uXXXX escape sequences, so the test source is pure ASCII and the assertion does not depend on the encoding used to compile it.

Verification

On JDK 17, default charset windows-1252:

• without the production change the new test fails with
  expected: <Abonnement Café — 5,99 € par mois> but was: <Abonnement Café — 5,99 € par mois>
• with it, ./gradlew clean test and ./gradlew javadoc both pass

Not included

Two related items I left out to keep the diff focused, happy to add them if you would prefer them here:

• the JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 line in ci-prb.yml is now redundant for compilation and could be dropped
• CI runs on a UTF-8 locale, so it would not catch a regression of this bug; a matrix entry running the tests with a non-UTF-8 default charset would lock the behavior in

I also did not touch CHANGELOG.md, since it appears to be updated by maintainers in the release commits.

[alexanderjordanbaker]

LGTM, thank you! Will be filing equivalent PRs on the other languages shortly

[matteobaccan]

The problem

SignedDataVerifier.parseJWTPayload converts the base64url-decoded payload bytes into a String without specifying a charset:

String payload = new String(Base64.getUrlDecoder().decode(jwt.getPayload()));

new String(byte[]) uses the JVM default charset. A JWT Claims Set is always a UTF-8 encoded JSON object (RFC 7519 §3, RFC 8259 §8.1), so the two only agree when the JVM happens to run with UTF-8 as its default.

That is not guaranteed on the Java versions this library supports. JEP 400 made UTF-8 the default only in Java 18; on Java 11 through 17 the default charset comes from the platform locale, so a server running on Windows or with a non-UTF-8 LANG gets windows-1252, ISO-8859-1, Shift_JIS, and so on.

When that happens, every non-ASCII character in a decoded payload is corrupted silently. The signature is verified against the raw JWS, so verification still succeeds and no exception is raised — the caller simply receives mojibake. This affects every entry point that goes through decodeSignedObject: verifyAndDecodeTransaction, verifyAndDecodeRenewalInfo, verifyAndDecodeNotification, verifyAndDecodeAppTransaction and verifyAndDecodeRealtimeRequest.

It is easy to reach in practice. The Advanced Commerce descriptors and items carry merchant-supplied free text (displayName, description), so any app that is not English-only can hit it.

Reproduction

On JDK 17 with windows-1252 as the default charset, decoding a signed transaction whose advancedCommerceInfo.descriptors.description is Abonnement Café — 5,99 € par mois returns:

Abonnement Café — 5,99 € par mois

The existing test suite cannot catch this: all of its payload fixtures are ASCII-only, and CI sets JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, which hides the difference.

The fix

new String(bytes, StandardCharsets.UTF_8) — one argument.

Source encoding

The build does not set options.encoding, so javac reads the sources with the platform charset too. 25 files under src/main contain non-ASCII characters (typographic apostrophes in the javadoc), which means those are corrupted in locally built javadoc on a non-UTF-8 machine. It also makes a regression test for this bug impossible to write in the natural way: the expected string literal would be corrupted by the compiler in exactly the same way as the actual value, and the assertion would pass.

This PR sets options.encoding = 'UTF-8' on the JavaCompile tasks and on javadoc. Note that ci-prb.yml currently compensates for the missing setting with JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, while ci-release-javadocs.yml does not set it at all.

The test

SignedDataVerifierTest.testNonAsciiDataDecodingIsIndependentOfTheDefaultCharset decodes a new fixture containing French and Japanese text. Its expected values are written as \uXXXX escape sequences, so the test source is pure ASCII and the assertion does not depend on the encoding used to compile it.

Verification

On JDK 17, default charset windows-1252:

• without the production change the new test fails with
  expected: <Abonnement Café — 5,99 € par mois> but was: <Abonnement Café — 5,99 € par mois>
• with it, ./gradlew clean test and ./gradlew javadoc both pass

Not included

Two related items I left out to keep the diff focused, happy to add them if you would prefer them here:

• the JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 line in ci-prb.yml is now redundant for compilation and could be dropped
• CI runs on a UTF-8 locale, so it would not catch a regression of this bug; a matrix entry running the tests with a non-UTF-8 default charset would lock the behavior in

I also did not touch CHANGELOG.md, since it appears to be updated by maintainers in the release commits.

[alexanderjordanbaker]

LGTM, thank you! Will be filing equivalent PRs on the other languages shortly

[matteobaccan]

The problem

SignedDataVerifier.parseJWTPayload converts the base64url-decoded payload bytes into a String without specifying a charset:

String payload = new String(Base64.getUrlDecoder().decode(jwt.getPayload()));

new String(byte[]) uses the JVM default charset. A JWT Claims Set is always a UTF-8 encoded JSON object (RFC 7519 §3, RFC 8259 §8.1), so the two only agree when the JVM happens to run with UTF-8 as its default.

That is not guaranteed on the Java versions this library supports. JEP 400 made UTF-8 the default only in Java 18; on Java 11 through 17 the default charset comes from the platform locale, so a server running on Windows or with a non-UTF-8 LANG gets windows-1252, ISO-8859-1, Shift_JIS, and so on.

When that happens, every non-ASCII character in a decoded payload is corrupted silently. The signature is verified against the raw JWS, so verification still succeeds and no exception is raised — the caller simply receives mojibake. This affects every entry point that goes through decodeSignedObject: verifyAndDecodeTransaction, verifyAndDecodeRenewalInfo, verifyAndDecodeNotification, verifyAndDecodeAppTransaction and verifyAndDecodeRealtimeRequest.

It is easy to reach in practice. The Advanced Commerce descriptors and items carry merchant-supplied free text (displayName, description), so any app that is not English-only can hit it.

Reproduction

On JDK 17 with windows-1252 as the default charset, decoding a signed transaction whose advancedCommerceInfo.descriptors.description is Abonnement Café — 5,99 € par mois returns:

Abonnement Café — 5,99 € par mois

The existing test suite cannot catch this: all of its payload fixtures are ASCII-only, and CI sets JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, which hides the difference.

The fix

new String(bytes, StandardCharsets.UTF_8) — one argument.

Source encoding

The build does not set options.encoding, so javac reads the sources with the platform charset too. 25 files under src/main contain non-ASCII characters (typographic apostrophes in the javadoc), which means those are corrupted in locally built javadoc on a non-UTF-8 machine. It also makes a regression test for this bug impossible to write in the natural way: the expected string literal would be corrupted by the compiler in exactly the same way as the actual value, and the assertion would pass.

This PR sets options.encoding = 'UTF-8' on the JavaCompile tasks and on javadoc. Note that ci-prb.yml currently compensates for the missing setting with JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, while ci-release-javadocs.yml does not set it at all.

The test

SignedDataVerifierTest.testNonAsciiDataDecodingIsIndependentOfTheDefaultCharset decodes a new fixture containing French and Japanese text. Its expected values are written as \uXXXX escape sequences, so the test source is pure ASCII and the assertion does not depend on the encoding used to compile it.

Verification

On JDK 17, default charset windows-1252:

• without the production change the new test fails with
  expected: <Abonnement Café — 5,99 € par mois> but was: <Abonnement Café — 5,99 € par mois>
• with it, ./gradlew clean test and ./gradlew javadoc both pass

Not included

Two related items I left out to keep the diff focused, happy to add them if you would prefer them here:

• the JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 line in ci-prb.yml is now redundant for compilation and could be dropped
• CI runs on a UTF-8 locale, so it would not catch a regression of this bug; a matrix entry running the tests with a non-UTF-8 default charset would lock the behavior in

I also did not touch CHANGELOG.md, since it appears to be updated by maintainers in the release commits.

[alexanderjordanbaker]

LGTM, thank you! Will be filing equivalent PRs on the other languages shortly

[matteobaccan]

The problem

SignedDataVerifier.parseJWTPayload converts the base64url-decoded payload bytes into a String without specifying a charset:

String payload = new String(Base64.getUrlDecoder().decode(jwt.getPayload()));

new String(byte[]) uses the JVM default charset. A JWT Claims Set is always a UTF-8 encoded JSON object (RFC 7519 §3, RFC 8259 §8.1), so the two only agree when the JVM happens to run with UTF-8 as its default.

That is not guaranteed on the Java versions this library supports. JEP 400 made UTF-8 the default only in Java 18; on Java 11 through 17 the default charset comes from the platform locale, so a server running on Windows or with a non-UTF-8 LANG gets windows-1252, ISO-8859-1, Shift_JIS, and so on.

When that happens, every non-ASCII character in a decoded payload is corrupted silently. The signature is verified against the raw JWS, so verification still succeeds and no exception is raised — the caller simply receives mojibake. This affects every entry point that goes through decodeSignedObject: verifyAndDecodeTransaction, verifyAndDecodeRenewalInfo, verifyAndDecodeNotification, verifyAndDecodeAppTransaction and verifyAndDecodeRealtimeRequest.

It is easy to reach in practice. The Advanced Commerce descriptors and items carry merchant-supplied free text (displayName, description), so any app that is not English-only can hit it.

Reproduction

On JDK 17 with windows-1252 as the default charset, decoding a signed transaction whose advancedCommerceInfo.descriptors.description is Abonnement Café — 5,99 € par mois returns:

Abonnement Café — 5,99 € par mois

The existing test suite cannot catch this: all of its payload fixtures are ASCII-only, and CI sets JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, which hides the difference.

The fix

new String(bytes, StandardCharsets.UTF_8) — one argument.

Source encoding

The build does not set options.encoding, so javac reads the sources with the platform charset too. 25 files under src/main contain non-ASCII characters (typographic apostrophes in the javadoc), which means those are corrupted in locally built javadoc on a non-UTF-8 machine. It also makes a regression test for this bug impossible to write in the natural way: the expected string literal would be corrupted by the compiler in exactly the same way as the actual value, and the assertion would pass.

This PR sets options.encoding = 'UTF-8' on the JavaCompile tasks and on javadoc. Note that ci-prb.yml currently compensates for the missing setting with JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, while ci-release-javadocs.yml does not set it at all.

The test

SignedDataVerifierTest.testNonAsciiDataDecodingIsIndependentOfTheDefaultCharset decodes a new fixture containing French and Japanese text. Its expected values are written as \uXXXX escape sequences, so the test source is pure ASCII and the assertion does not depend on the encoding used to compile it.

Verification

On JDK 17, default charset windows-1252:

• without the production change the new test fails with
  expected: <Abonnement Café — 5,99 € par mois> but was: <Abonnement Café — 5,99 € par mois>
• with it, ./gradlew clean test and ./gradlew javadoc both pass

Not included

Two related items I left out to keep the diff focused, happy to add them if you would prefer them here:

• the JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 line in ci-prb.yml is now redundant for compilation and could be dropped
• CI runs on a UTF-8 locale, so it would not catch a regression of this bug; a matrix entry running the tests with a non-UTF-8 default charset would lock the behavior in

I also did not touch CHANGELOG.md, since it appears to be updated by maintainers in the release commits.

[alexanderjordanbaker]

LGTM, thank you! Will be filing equivalent PRs on the other languages shortly

[matteobaccan]

The problem

SignedDataVerifier.parseJWTPayload converts the base64url-decoded payload bytes into a String without specifying a charset:

String payload = new String(Base64.getUrlDecoder().decode(jwt.getPayload()));

new String(byte[]) uses the JVM default charset. A JWT Claims Set is always a UTF-8 encoded JSON object (RFC 7519 §3, RFC 8259 §8.1), so the two only agree when the JVM happens to run with UTF-8 as its default.

That is not guaranteed on the Java versions this library supports. JEP 400 made UTF-8 the default only in Java 18; on Java 11 through 17 the default charset comes from the platform locale, so a server running on Windows or with a non-UTF-8 LANG gets windows-1252, ISO-8859-1, Shift_JIS, and so on.

When that happens, every non-ASCII character in a decoded payload is corrupted silently. The signature is verified against the raw JWS, so verification still succeeds and no exception is raised — the caller simply receives mojibake. This affects every entry point that goes through decodeSignedObject: verifyAndDecodeTransaction, verifyAndDecodeRenewalInfo, verifyAndDecodeNotification, verifyAndDecodeAppTransaction and verifyAndDecodeRealtimeRequest.

It is easy to reach in practice. The Advanced Commerce descriptors and items carry merchant-supplied free text (displayName, description), so any app that is not English-only can hit it.

Reproduction

On JDK 17 with windows-1252 as the default charset, decoding a signed transaction whose advancedCommerceInfo.descriptors.description is Abonnement Café — 5,99 € par mois returns:

Abonnement Café — 5,99 € par mois

The existing test suite cannot catch this: all of its payload fixtures are ASCII-only, and CI sets JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, which hides the difference.

The fix

new String(bytes, StandardCharsets.UTF_8) — one argument.

Source encoding

The build does not set options.encoding, so javac reads the sources with the platform charset too. 25 files under src/main contain non-ASCII characters (typographic apostrophes in the javadoc), which means those are corrupted in locally built javadoc on a non-UTF-8 machine. It also makes a regression test for this bug impossible to write in the natural way: the expected string literal would be corrupted by the compiler in exactly the same way as the actual value, and the assertion would pass.

This PR sets options.encoding = 'UTF-8' on the JavaCompile tasks and on javadoc. Note that ci-prb.yml currently compensates for the missing setting with JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, while ci-release-javadocs.yml does not set it at all.

The test

SignedDataVerifierTest.testNonAsciiDataDecodingIsIndependentOfTheDefaultCharset decodes a new fixture containing French and Japanese text. Its expected values are written as \uXXXX escape sequences, so the test source is pure ASCII and the assertion does not depend on the encoding used to compile it.

Verification

On JDK 17, default charset windows-1252:

• without the production change the new test fails with
  expected: <Abonnement Café — 5,99 € par mois> but was: <Abonnement Café — 5,99 € par mois>
• with it, ./gradlew clean test and ./gradlew javadoc both pass

Not included

Two related items I left out to keep the diff focused, happy to add them if you would prefer them here:

• the JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 line in ci-prb.yml is now redundant for compilation and could be dropped
• CI runs on a UTF-8 locale, so it would not catch a regression of this bug; a matrix entry running the tests with a non-UTF-8 default charset would lock the behavior in

I also did not touch CHANGELOG.md, since it appears to be updated by maintainers in the release commits.

[alexanderjordanbaker]

LGTM, thank you! Will be filing equivalent PRs on the other languages shortly

[matteobaccan]

The problem

SignedDataVerifier.parseJWTPayload converts the base64url-decoded payload bytes into a String without specifying a charset:

String payload = new String(Base64.getUrlDecoder().decode(jwt.getPayload()));

new String(byte[]) uses the JVM default charset. A JWT Claims Set is always a UTF-8 encoded JSON object (RFC 7519 §3, RFC 8259 §8.1), so the two only agree when the JVM happens to run with UTF-8 as its default.

That is not guaranteed on the Java versions this library supports. JEP 400 made UTF-8 the default only in Java 18; on Java 11 through 17 the default charset comes from the platform locale, so a server running on Windows or with a non-UTF-8 LANG gets windows-1252, ISO-8859-1, Shift_JIS, and so on.

When that happens, every non-ASCII character in a decoded payload is corrupted silently. The signature is verified against the raw JWS, so verification still succeeds and no exception is raised — the caller simply receives mojibake. This affects every entry point that goes through decodeSignedObject: verifyAndDecodeTransaction, verifyAndDecodeRenewalInfo, verifyAndDecodeNotification, verifyAndDecodeAppTransaction and verifyAndDecodeRealtimeRequest.

It is easy to reach in practice. The Advanced Commerce descriptors and items carry merchant-supplied free text (displayName, description), so any app that is not English-only can hit it.

Reproduction

On JDK 17 with windows-1252 as the default charset, decoding a signed transaction whose advancedCommerceInfo.descriptors.description is Abonnement Café — 5,99 € par mois returns:

Abonnement Café — 5,99 € par mois

The existing test suite cannot catch this: all of its payload fixtures are ASCII-only, and CI sets JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, which hides the difference.

The fix

new String(bytes, StandardCharsets.UTF_8) — one argument.

Source encoding

The build does not set options.encoding, so javac reads the sources with the platform charset too. 25 files under src/main contain non-ASCII characters (typographic apostrophes in the javadoc), which means those are corrupted in locally built javadoc on a non-UTF-8 machine. It also makes a regression test for this bug impossible to write in the natural way: the expected string literal would be corrupted by the compiler in exactly the same way as the actual value, and the assertion would pass.

This PR sets options.encoding = 'UTF-8' on the JavaCompile tasks and on javadoc. Note that ci-prb.yml currently compensates for the missing setting with JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, while ci-release-javadocs.yml does not set it at all.

The test

SignedDataVerifierTest.testNonAsciiDataDecodingIsIndependentOfTheDefaultCharset decodes a new fixture containing French and Japanese text. Its expected values are written as \uXXXX escape sequences, so the test source is pure ASCII and the assertion does not depend on the encoding used to compile it.

Verification

On JDK 17, default charset windows-1252:

• without the production change the new test fails with
  expected: <Abonnement Café — 5,99 € par mois> but was: <Abonnement Café — 5,99 € par mois>
• with it, ./gradlew clean test and ./gradlew javadoc both pass

Not included

Two related items I left out to keep the diff focused, happy to add them if you would prefer them here:

• the JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 line in ci-prb.yml is now redundant for compilation and could be dropped
• CI runs on a UTF-8 locale, so it would not catch a regression of this bug; a matrix entry running the tests with a non-UTF-8 default charset would lock the behavior in

I also did not touch CHANGELOG.md, since it appears to be updated by maintainers in the release commits.

[alexanderjordanbaker]

LGTM, thank you! Will be filing equivalent PRs on the other languages shortly

[matteobaccan]

The problem

SignedDataVerifier.parseJWTPayload converts the base64url-decoded payload bytes into a String without specifying a charset:

String payload = new String(Base64.getUrlDecoder().decode(jwt.getPayload()));

new String(byte[]) uses the JVM default charset. A JWT Claims Set is always a UTF-8 encoded JSON object (RFC 7519 §3, RFC 8259 §8.1), so the two only agree when the JVM happens to run with UTF-8 as its default.

That is not guaranteed on the Java versions this library supports. JEP 400 made UTF-8 the default only in Java 18; on Java 11 through 17 the default charset comes from the platform locale, so a server running on Windows or with a non-UTF-8 LANG gets windows-1252, ISO-8859-1, Shift_JIS, and so on.

When that happens, every non-ASCII character in a decoded payload is corrupted silently. The signature is verified against the raw JWS, so verification still succeeds and no exception is raised — the caller simply receives mojibake. This affects every entry point that goes through decodeSignedObject: verifyAndDecodeTransaction, verifyAndDecodeRenewalInfo, verifyAndDecodeNotification, verifyAndDecodeAppTransaction and verifyAndDecodeRealtimeRequest.

It is easy to reach in practice. The Advanced Commerce descriptors and items carry merchant-supplied free text (displayName, description), so any app that is not English-only can hit it.

Reproduction

On JDK 17 with windows-1252 as the default charset, decoding a signed transaction whose advancedCommerceInfo.descriptors.description is Abonnement Café — 5,99 € par mois returns:

Abonnement Café — 5,99 € par mois

The existing test suite cannot catch this: all of its payload fixtures are ASCII-only, and CI sets JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, which hides the difference.

The fix

new String(bytes, StandardCharsets.UTF_8) — one argument.

Source encoding

The build does not set options.encoding, so javac reads the sources with the platform charset too. 25 files under src/main contain non-ASCII characters (typographic apostrophes in the javadoc), which means those are corrupted in locally built javadoc on a non-UTF-8 machine. It also makes a regression test for this bug impossible to write in the natural way: the expected string literal would be corrupted by the compiler in exactly the same way as the actual value, and the assertion would pass.

This PR sets options.encoding = 'UTF-8' on the JavaCompile tasks and on javadoc. Note that ci-prb.yml currently compensates for the missing setting with JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, while ci-release-javadocs.yml does not set it at all.

The test

SignedDataVerifierTest.testNonAsciiDataDecodingIsIndependentOfTheDefaultCharset decodes a new fixture containing French and Japanese text. Its expected values are written as \uXXXX escape sequences, so the test source is pure ASCII and the assertion does not depend on the encoding used to compile it.

Verification

On JDK 17, default charset windows-1252:

• without the production change the new test fails with
  expected: <Abonnement Café — 5,99 € par mois> but was: <Abonnement Café — 5,99 € par mois>
• with it, ./gradlew clean test and ./gradlew javadoc both pass

Not included

Two related items I left out to keep the diff focused, happy to add them if you would prefer them here:

• the JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 line in ci-prb.yml is now redundant for compilation and could be dropped
• CI runs on a UTF-8 locale, so it would not catch a regression of this bug; a matrix entry running the tests with a non-UTF-8 default charset would lock the behavior in

I also did not touch CHANGELOG.md, since it appears to be updated by maintainers in the release commits.

[alexanderjordanbaker]

LGTM, thank you! Will be filing equivalent PRs on the other languages shortly

[matteobaccan]

The problem

SignedDataVerifier.parseJWTPayload converts the base64url-decoded payload bytes into a String without specifying a charset:

String payload = new String(Base64.getUrlDecoder().decode(jwt.getPayload()));

new String(byte[]) uses the JVM default charset. A JWT Claims Set is always a UTF-8 encoded JSON object (RFC 7519 §3, RFC 8259 §8.1), so the two only agree when the JVM happens to run with UTF-8 as its default.

That is not guaranteed on the Java versions this library supports. JEP 400 made UTF-8 the default only in Java 18; on Java 11 through 17 the default charset comes from the platform locale, so a server running on Windows or with a non-UTF-8 LANG gets windows-1252, ISO-8859-1, Shift_JIS, and so on.

When that happens, every non-ASCII character in a decoded payload is corrupted silently. The signature is verified against the raw JWS, so verification still succeeds and no exception is raised — the caller simply receives mojibake. This affects every entry point that goes through decodeSignedObject: verifyAndDecodeTransaction, verifyAndDecodeRenewalInfo, verifyAndDecodeNotification, verifyAndDecodeAppTransaction and verifyAndDecodeRealtimeRequest.

It is easy to reach in practice. The Advanced Commerce descriptors and items carry merchant-supplied free text (displayName, description), so any app that is not English-only can hit it.

Reproduction

On JDK 17 with windows-1252 as the default charset, decoding a signed transaction whose advancedCommerceInfo.descriptors.description is Abonnement Café — 5,99 € par mois returns:

Abonnement Café — 5,99 € par mois

The existing test suite cannot catch this: all of its payload fixtures are ASCII-only, and CI sets JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, which hides the difference.

The fix

new String(bytes, StandardCharsets.UTF_8) — one argument.

Source encoding

The build does not set options.encoding, so javac reads the sources with the platform charset too. 25 files under src/main contain non-ASCII characters (typographic apostrophes in the javadoc), which means those are corrupted in locally built javadoc on a non-UTF-8 machine. It also makes a regression test for this bug impossible to write in the natural way: the expected string literal would be corrupted by the compiler in exactly the same way as the actual value, and the assertion would pass.

This PR sets options.encoding = 'UTF-8' on the JavaCompile tasks and on javadoc. Note that ci-prb.yml currently compensates for the missing setting with JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8, while ci-release-javadocs.yml does not set it at all.

The test

SignedDataVerifierTest.testNonAsciiDataDecodingIsIndependentOfTheDefaultCharset decodes a new fixture containing French and Japanese text. Its expected values are written as \uXXXX escape sequences, so the test source is pure ASCII and the assertion does not depend on the encoding used to compile it.

Verification

On JDK 17, default charset windows-1252:

• without the production change the new test fails with
  expected: <Abonnement Café — 5,99 € par mois> but was: <Abonnement Café — 5,99 € par mois>
• with it, ./gradlew clean test and ./gradlew javadoc both pass

Not included

Two related items I left out to keep the diff focused, happy to add them if you would prefer them here:

• the JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 line in ci-prb.yml is now redundant for compilation and could be dropped
• CI runs on a UTF-8 locale, so it would not catch a regression of this bug; a matrix entry running the tests with a non-UTF-8 default charset would lock the behavior in

I also did not touch CHANGELOG.md, since it appears to be updated by maintainers in the release commits.

[alexanderjordanbaker]

LGTM, thank you! Will be filing equivalent PRs on the other languages shortly