Uh oh!
There was an error while loading. Please reload this page.
SnowflakeIO yaml - #39742
Conversation
| import org.apache.beam.sdk.values.Row; | ||
| /** A {@link SchemaTransformProvider} for reading rows from Snowflake. */ | ||
| @SuppressWarnings({ |
There was a problem hiding this comment.
Prefer not introducing SuppressWarnings for new code
| }) | ||
| public class SnowflakeSchemaTransformUtils { | ||
| public static SnowflakeIO.DataSourceConfiguration createDataSourceConfiguration( |
There was a problem hiding this comment.
Why not just make SnowflakeReadSchemaTransformProvider.Configuration as createDataSourceConfiguration's parameter?
There was a problem hiding this comment.
It is shared by both the read and write providers, which have different configuration classes. Passing the individual fields keeps this utility independent of either provider-specific configuration.
| String debugMode = getDebugMode(); | ||
| if (debugMode != null) { | ||
| parseStreamingLogLevel(debugMode); |
There was a problem hiding this comment.
return value is unused. I understand it's for validation. Consider add a comment noting it.
Assigning reviewers: R: @claudevdm for label python. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
derrickaw
commented
Aug 18, 2026
Hi @Amar3tto, can you add a yaml pipeline test under yaml/tests or yaml/extended_tests with any improvements necessary in integration_tests.py? Thanks. |
Amar3tto
commented
Aug 20, 2026
Added a Snowflake YAML pipeline test under |
| dependsOn ":sdks:java:io:expansion-service:build" | ||
| dependsOn ":sdks:java:io:google-cloud-platform:expansion-service:build" | ||
| dependsOn ":sdks:java:io:debezium:expansion-service:shadowJar" | ||
| dependsOn ":sdks:java:io:snowflake:expansion-service:shadowJar" |
There was a problem hiding this comment.
Consider move this to postCommitYamlIntegrationTests. The test only need to be exercised on postsubmit time. The PreCommit becomes heavier as we are adding more expansion services (and test container). Same apply to kenesis / JmsIO / debezium
Fixes#28691
Adds Beam YAML support for Snowflake IO.
ReadFromSnowflakefor batch reads from a table or query.WriteToSnowflakefor batch writes and streaming writes via Snowpipe.Row/ Snowflake schema conversion.standard_io.yaml.https://github.com/apache/beam/actions/runs/31692701363
https://github.com/apache/beam/actions/runs/31691059139
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.