Skip to content
This repository was archived by the owner on Mar 6, 2026. It is now read-only.

feat: Adds source_column_match and associated tests - #2227

Merged
gcf-merge-on-green[bot] merged 14 commits into
mainfrom
feat-374142081-add-source-column-match
Jul 15, 2025
Merged

feat: Adds source_column_match and associated tests#2227
gcf-merge-on-green[bot] merged 14 commits into
mainfrom
feat-374142081-add-source-column-match

Conversation

@chalmerlowe

@chalmerlowechalmerlowe commented Jul 2, 2025

Copy link
Copy Markdown
Collaborator

This commit introduces new configuration options for BigQuery load jobs and external table definitions, aligning with recent updates to the underlying protos.

New options added:

  • source_column_name_match_option: Controls how source columns are matched to the schema. (Applies to LoadJobConfig, ExternalConfig, LoadJob)

Changes include:

  • Added corresponding properties (getters/setters) to LoadJobConfig, LoadJob, ExternalConfig, and CSVOptions.
  • Updated docstrings and type hints for all new attributes.
  • Updated unit tests to cover the new options, ensuring they are correctly handled during object initialization, serialization to API representation, and deserialization from API responses.

@chalmerlowe
chalmerlowe requested review from a team and agrawal-siddharthJuly 2, 2025 16:22
@product-auto-labelproduct-auto-labelBot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/python-bigquery API. labels Jul 2, 2025
@chalmerlowechalmerlowe assigned Linchin and unassigned yirutangJul 2, 2025
@chalmerlowe
chalmerlowe requested review from Linchin and removed request for agrawal-siddharthJuly 2, 2025 16:23
@chalmerlowechalmerlowe self-assigned this Jul 2, 2025
chalmerlowe
chalmerlowe commented Jul 2, 2025
View reviewed changes
Comment threadgoogle/cloud/bigquery/job/load.py
chalmerlowe
chalmerlowe commented Jul 2, 2025
View reviewed changes
Comment threadgoogle/cloud/bigquery/external_config.py
chalmerlowe
chalmerlowe commented Jul 2, 2025
View reviewed changes

@chalmerlowechalmerlowe left a comment

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Altered docstring formatting to try and eliminate a docs error.

Comment threadgoogle/cloud/bigquery/job/load.py Outdated
Comment threadgoogle/cloud/bigquery/external_config.py Outdated
@chalmerlowechalmerlowe added the automerge Merge the pull request once unit tests and other checks pass. label Jul 15, 2025
tswast
tswast reviewed Jul 15, 2025
View reviewed changes
Comment threadgoogle/cloud/bigquery/external_config.py Outdated
Comment threadgoogle/cloud/bigquery/external_config.py Outdated
# return None

@source_column_match.setter
def source_column_match(self, value: Optional[SourceColumnMatch]):

@tswasttswastJul 15, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like us to allow str in the setter as well. Can you update this, please?

Comment threadgoogle/cloud/bigquery/job/load.py Outdated
return None

@source_column_match.setter
def source_column_match(self, value: Optional[SourceColumnMatch]):

@tswasttswastJul 15, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. Can we allow str, please?

Comment threadgoogle/cloud/bigquery/job/load.py Outdated
return self.configuration.clustering_fields

@property
def source_column_match(self):

@tswasttswastJul 15, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing return type annotation?

Comment threadtests/unit/job/test_load_config.py
config._properties["load"]["sourceColumnMatch"] = option_enum.value
self.assertEqual(config.source_column_match, option_enum)

def test_source_column_match_setter(self):

@tswasttswastJul 15, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add another test where we use the setter with a str, too?

@chalmerlowechalmerloweJul 15, 2025•
edited
Loading

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, both here and in test_external_config.py

tswast
tswast approved these changes Jul 15, 2025
View reviewed changes

@tswasttswast left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@gcf-merge-on-green
gcf-merge-on-greenBot merged commit 6d5d236 into mainJul 15, 2025
28 checks passed
@gcf-merge-on-greengcf-merge-on-greenBot removed the automerge Merge the pull request once unit tests and other checks pass. label Jul 15, 2025
@gcf-merge-on-green
gcf-merge-on-greenBot deleted the feat-374142081-add-source-column-match branch July 15, 2025 22:04
@release-pleaserelease-pleaseBot mentioned this pull request Jul 15, 2025
Merged
This was referenced Jul 22, 2025
Closed
Closed
Closed
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@LinchinLinchinAwaiting requested review from Linchin
1 more reviewer
@tswasttswasttswast approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@chalmerlowechalmerlowe

@LinchinLinchin

Labels

api: bigqueryIssues related to the googleapis/python-bigquery API.size: mPull request size is medium.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@chalmerlowe@tswast@Linchin@yirutang