Skip to content
This repository was archived by the owner on Mar 6, 2026. It is now read-only.

feature: raise error for unknown properties in job config - #446

Merged
tswast merged 6 commits into
googleapis:masterfrom
cguardia:303-unknown-properties
Jan 13, 2021
Merged

feature: raise error for unknown properties in job config#446
tswast merged 6 commits into
googleapis:masterfrom
cguardia:303-unknown-properties

Conversation

@cguardia

@cguardiacguardia commented Dec 22, 2020•
edited
Loading

Copy link
Copy Markdown
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Simple feature to raise warning if unknown property is set on job_config instances.

Fixes#303 🦕

@cguardia
cguardia requested review from a team and tswast and removed request for a teamDecember 22, 2020 09:51
@product-auto-labelproduct-auto-labelBot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Dec 22, 2020
@google-clagoogle-claBot added the cla: yes This human has signed the Contributor License Agreement. label Dec 22, 2020
tswast
tswast reviewed Jan 7, 2021
View reviewed changes
Comment threadgoogle/cloud/bigquery/job/base.py Outdated
def __setattr__(self, name, value):
"""Override to be able to warn if an uknown property is being set"""
if not name.startswith("_") and name not in type(self).__dict__:
warnings.warn("Property {} is unknown for {}.".format(name, type(self)))

@tswasttswastJan 7, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, I think we can go a step further and throw AttributeError, just like object() instances do.

@cguardiacguardiaJan 8, 2021

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Done.

@cguardiacguardia changed the title feature: warn about unknown properties in job configfeature: raise error for unknown properties in job configJan 8, 2021
tswast
tswast approved these changes Jan 8, 2021
View reviewed changes

@tswasttswast left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@tswast

tswast commented Jan 8, 2021

Copy link
Copy Markdown
Contributor
 def __setattr__(self, name, value):
"""Override to be able to raise error if an unknown property is being set"""
if not name.startswith("_") and name not in type(self).__dict__:
> raise AttributeError("Property {} is unknown for {}.".format(name, type(self)))
E AttributeError: Property labels is unknown for <class 'google.cloud.bigquery.job.query.QueryJobConfig'>.

@tswast

tswast commented Jan 8, 2021

Copy link
Copy Markdown
Contributor

It appears you've uncovered a bug in the client. I don't see a setter for labels!

python-bigquery/google/cloud/bigquery/job/base.py

Line 234 in 0337ea0

deflabels(self):

@tswast

tswast commented Jan 8, 2021

Copy link
Copy Markdown
Contributor

Nevermind, there is a setter:

python-bigquery/google/cloud/bigquery/job/base.py

Line 675 in 0337ea0

@labels.setter

Perhaps this logic needs to be adjusted in order to account for properties defined in the superclass?

tswast
tswast suggested changes Jan 8, 2021
View reviewed changes
Comment threadgoogle/cloud/bigquery/job/base.py Outdated

def __setattr__(self, name, value):
"""Override to be able to raise error if an unknown property is being set"""
if not name.startswith("_") and name not in type(self).__dict__:

@tswasttswastJan 8, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try using hasattr to see if that accounts for superclass properties.

Suggested change
ifnotname.startswith("_") andnamenotintype(self).__dict__:
ifnotname.startswith("_") andnothasattr(self, name):

@cguardiacguardiaJan 10, 2021

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, but we have to use type(self), because if not the properties are looked up on the instance.

@cguardia
cguardia requested a review from tswastJanuary 12, 2021 19:29
@tswast

tswast commented Jan 13, 2021

Copy link
Copy Markdown
Contributor

One of our system tests actually had a mistake, which this change caught. There is no "destination" property on LoadJobConfig https://googleapis.dev/python/bigquery/latest/generated/google.cloud.bigquery.job.LoadJobConfig.html

@plamut

plamut commented Jan 13, 2021•
edited
Loading

Copy link
Copy Markdown
Contributor

Re: labels error - as discussed offline, it's actually a bug in the base _AsyncJob.

@tswast
tswast merged commit 1526e39 into googleapis:masterJan 13, 2021
@release-pleaserelease-pleaseBot mentioned this pull request Jan 4, 2022
Closed
evansd added a commit to bennettoxford/openprescribing that referenced this pull request Dec 2, 2022
@evansd
Don't set location attribute on CopyJobConfig
f329a4a
As far as I can tell, this has never been a valid config value here. But
prior to the merging of [this PR][1] in v2.7.0, invalid attributes were
silently ignored.
[1]: googleapis/python-bigquery#446
evansd added a commit to bennettoxford/openprescribing that referenced this pull request Dec 2, 2022
@evansd
Don't set location attribute on CopyJobConfig
23b3476
As far as I can tell, this has never been a valid config value here. But
prior to the merging of [this PR][1] in v2.7.0, invalid attributes were
silently ignored.
[1]: googleapis/python-bigquery#446
@evansdevansd mentioned this pull request Dec 2, 2022
Merged
evansd added a commit to bennettoxford/openprescribing that referenced this pull request Dec 2, 2022
@evansd
Don't set location attribute on CopyJobConfig
f3f68cc
As far as I can tell, this has never been a valid config value here. But
prior to the merging of [this PR][1] in v2.7.0, invalid attributes were
silently ignored.
[1]: googleapis/python-bigquery#446
evansd added a commit to bennettoxford/openprescribing that referenced this pull request Dec 2, 2022
@evansd
Don't set location attribute on CopyJobConfig
b0e259b
As far as I can tell, this has never been a valid config value here. But
prior to the merging of [this PR][1] in v2.7.0, invalid attributes were
silently ignored.
[1]: googleapis/python-bigquery#446
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

1 more reviewer
@tswasttswasttswast approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

api: bigqueryIssues related to the googleapis/python-bigquery API.cla: yesThis human has signed the Contributor License Agreement.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Resource objects such as *JobConfig do not error or warn when setting unknown properties

3 participants

@cguardia@tswast@plamut