Update gcp bom to 26.50.0

[damccorm]

This notably includes upgrading protobuf. Changes are segmented by commit:

• Update the bom with the bom updater script - e8b2c0a
• Remove null checks - 5762104 - this is because proto fields can no longer be null (they'll at least have a default implementation)
• Update google_cloud_bigdataoss_version - 4a5efdd - this is because gcsio was not compatible with Protobuf 4 until this release.
• Update aws client versions to be compatible with Protobuf 4 - b72d791

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

[damccorm]

Some failures are caused because proto fields can no longer be null (they'll at least have a default implementation). Addressing this now

[github-actions]

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @m-trieu for label java.
R: @Abacn for label build.
R: @shunping for label io.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[damccorm]

cc/ @ldetmer

[shunping]

LGTM

[Abacn]

It seems this breaks https://github.com/apache/beam/actions/workflows/beam_PostCommit_Java_Hadoop_Versions.yml?query=event%3Aschedule: #33252

[Abacn]

and https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Debezium_IO_Direct.yml?query=event%3Aschedule also

[damccorm]

Thanks - #33525 should fix the first issue, working on a fix for the second

[Abacn]

Thanks, persumably debezium test is the same cause? (which could be trickier as we have conflict protobuf version for DebeziumIO then)

[damccorm]

Yeah, I'm seeing if I can upgrade our debezium dependency version. That does seem like a larger issue though

[damccorm]

One thing which may help is that the problematic dependency is actually a test-only dependency -

beam/sdks/java/io/debezium/build.gradle

Line 65 in 4a5575c

testImplementation group: 'io.debezium', name: 'debezium-connector-postgres', version: '1.3.1.Final'

- debezium-connector-postgres

So worst case, we could probably only force the dependency for testing postgres, but that is definitely not an awesome place to land...

[clairemcginty]

hi! I was wondering if there were plans to bump the GrpcVendoring protobuf version to 4.x as well? I also noticed when updating beam-sdks-java-extensions-protobuf to 2.63, it's transitively pulling protobuf-java 4.29 but protobuf-java-util 3.25:

% cs resolve org.apache.beam:beam-sdks-java-extensions-protobuf:2.63.0
args4j:args4j:2.33:default
com.charleskorn.kaml:kaml:0.20.0:default
com.fasterxml.jackson.core:jackson-annotations:2.15.4:default
com.fasterxml.jackson.core:jackson-core:2.16.1:default
com.fasterxml.jackson.core:jackson-databind:2.15.4:default
...
com.google.protobuf:protobuf-java:4.29.0:default
com.google.protobuf:protobuf-java-util:3.25.2:default

and beam-sdks-java-extensions-google-cloud-platform-core only pulls in 3.25:

% cs resolve org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.63.0
args4j:args4j:2.33:default
com.fasterxml.jackson.core:jackson-annotations:2.15.4:default
com.fasterxml.jackson.core:jackson-core:2.18.0:default
com.fasterxml.jackson.core:jackson-databind:2.15.4:default
...
com.google.protobuf:protobuf-java:3.25.5:default
com.google.protobuf:protobuf-java-util:3.25.5:default

[damccorm]

hi! I was wondering if there were plans to bump the GrpcVendoring protobuf version to 4.x as well? I also noticed when updating beam-sdks-java-extensions-protobuf to 2.63, it's transitively pulling protobuf-java 4.29 but protobuf-java-util 3.25:

% cs resolve org.apache.beam:beam-sdks-java-extensions-protobuf:2.63.0
args4j:args4j:2.33:default
com.charleskorn.kaml:kaml:0.20.0:default
com.fasterxml.jackson.core:jackson-annotations:2.15.4:default
com.fasterxml.jackson.core:jackson-core:2.16.1:default
com.fasterxml.jackson.core:jackson-databind:2.15.4:default
...
com.google.protobuf:protobuf-java:4.29.0:default
com.google.protobuf:protobuf-java-util:3.25.2:default

and beam-sdks-java-extensions-google-cloud-platform-core only pulls in 3.25:

% cs resolve org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.63.0
args4j:args4j:2.33:default
com.fasterxml.jackson.core:jackson-annotations:2.15.4:default
com.fasterxml.jackson.core:jackson-core:2.18.0:default
com.fasterxml.jackson.core:jackson-databind:2.15.4:default
...
com.google.protobuf:protobuf-java:3.25.5:default
com.google.protobuf:protobuf-java-util:3.25.5:default

That dependency comes from https://github.com/grpc/grpc-java/blob/34a7cffcf3dca012cd3e48da14d127c3afd924bf/gradle/libs.versions.toml#L9 so we would either need to wait for a new grpc release or we'd need to override this version (which may have unknown consequences). Because of this setup and the requirements of the Google BOM, historically, we have not kept these versions in sync.

3.25.x is theoretically supposed to have some compatibility guarantees with 4.x so that they can work together, is that specifically giving you problems?

[clairemcginty]

thanks @damccorm! I haven't encountered any specific problems yet -- I was just concerned about compatibility if the GCP libraries BOM is on Protobuf 4 and Beam is mostly on 3. Good to know they're theoretically compatible 👍

[tkaymak]

The new Debezium 3.1 should have a newer proto version available. Would it now be feasible to upgrade DebeziumIO to use 3.1? (https://debezium.io/releases/3.1/release-notes#other_changes_5 -> https://github.com/debezium/container-images/pull/427/files)?