Skip to content

Cherry-pick: Add driverJars parameter to JdbcIO. (#25824) - #26474

Merged
jrmccluskey merged 2 commits into
apache:release-2.47.0from
bvolpato:cp-jdbc-io
Apr 30, 2023
Merged

Cherry-pick: Add driverJars parameter to JdbcIO. (#25824)#26474
jrmccluskey merged 2 commits into
apache:release-2.47.0from
bvolpato:cp-jdbc-io

Conversation

@bvolpato

@bvolpatobvolpato commented Apr 28, 2023

Copy link
Copy Markdown
Contributor

Cherry-picking #25824 to the release.

@pranavbhandari24Bruno Volpato
Add driverJars parameter to JdbcIO. (apache#25824)
0762aad
This change allows users to use driver jars saved in GCS. With this change, Dataflow templates will be able to migrate to JdbcIO instead of DynamicJdbcIO.
@bvolpato

bvolpato commented Apr 28, 2023

Copy link
Copy Markdown
ContributorAuthor

R: @jrmccluskey

@github-actions

github-actionsBot commented Apr 28, 2023

Copy link
Copy Markdown
Contributor

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control

@codecov

codecovBot commented Apr 28, 2023•
edited
Loading

Copy link
Copy Markdown

Codecov Report

Merging #26474 (dbaa8dc) into release-2.47.0 (4d1f1f1) will decrease coverage by 0.62%.
The diff coverage is n/a.

@@ Coverage Diff @@## release-2.47.0 #26474 +/- ##
==================================================
- Coverage 71.17% 70.55% -0.62% 
==================================================
Files 787 787 Lines 103294 103311 +17 ==================================================
- Hits 73518 72896 -622 - Misses 28287 28926 +639 
Partials 1489 1489
FlagCoverage Δ
python78.94% <ø> (-0.93%)⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted FilesCoverage Δ
sdks/python/apache_beam/io/jdbc.py79.16% <ø> (ø)

... and 29 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@jrmccluskey

jrmccluskey commented Apr 28, 2023

Copy link
Copy Markdown
Contributor

Run Python_Integration PreCommit

@Abacn

Abacn commented Apr 28, 2023•
edited
Loading

Copy link
Copy Markdown
Contributor

Possibly related: https://ci-beam.apache.org/view/PostCommit/job/beam_PostCommit_Python311/105/

Update: yes it is related. Kicked off the test in this cherry-pick PR and test also fails: https://ci-beam.apache.org/job/beam_PostCommit_Python311_PR/17/consoleFull

15:24:41 FAILED [ 75%]
15:24:41 apache_beam/io/external/xlang_jdbcio_it_test.py::CrossLanguageJdbcIOTest::test_xlang_jdbc_write_read_1_mysql

@Abacn

Abacn commented Apr 28, 2023

Copy link
Copy Markdown
Contributor

Run Python 3.11 PostCommit

Abacn
Abacn requested changes Apr 28, 2023
View reviewed changes
Comment threadsdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcSchemaIOProvider.java Outdated
@AbacnBruno Volpato
Fix jdbc xlang schema type mismatch (apache#26480)
dbaa8dc
* Fix jdbc xlang schema type mismatch
* Also fix fetch_size type mismatch
* Add new fields in the end
@bvolpato

bvolpato commented Apr 29, 2023

Copy link
Copy Markdown
ContributorAuthor

Updated with patch provided by @Abacn at #25062. Thanks Yi.

@bvolpato
bvolpato requested a review from AbacnApril 29, 2023 01:51
@Abacn

Abacn commented Apr 29, 2023

Copy link
Copy Markdown
Contributor

Run Python 3.11 PostCommit

Abacn
Abacn approved these changes Apr 29, 2023
View reviewed changes

@AbacnAbacn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM

@jrmccluskey

jrmccluskey commented Apr 29, 2023

Copy link
Copy Markdown
Contributor

Run Python 3.11 PostCommit

@jrmccluskey

jrmccluskey commented Apr 29, 2023

Copy link
Copy Markdown
Contributor

Run Python_Integration PreCommit

@jrmccluskey

jrmccluskey commented Apr 29, 2023

Copy link
Copy Markdown
Contributor

I'm not anticipating this PR significantly impacting the Python tests I'm re-running, will merge after those have any result

@Abacn

Abacn commented Apr 29, 2023

Copy link
Copy Markdown
Contributor

Dataflow tests are flaky recently due to no worker available in us-central1 region.

@jrmccluskey

jrmccluskey commented Apr 29, 2023

Copy link
Copy Markdown
Contributor

That would explain the flakey failures I've been seeing on streaming integration tests. Thank you for the context!

@jrmccluskey
jrmccluskey merged commit 0f12dfd into apache:release-2.47.0Apr 30, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AbacnAbacnAbacn approved these changes

Assignees

No one assigned

Labels

iojavajdbcpython

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@bvolpato@jrmccluskey@Abacn@pranavbhandari24