Skip to content
This repository was archived by the owner on Mar 6, 2026. It is now read-only.

fix: raise error if inserting rows with unknown fields - #163

Merged
tseaver merged 7 commits into
googleapis:masterfrom
plamut:iss-151
Jul 30, 2020
Merged

fix: raise error if inserting rows with unknown fields#163
tseaver merged 7 commits into
googleapis:masterfrom
plamut:iss-151

Conversation

@plamut

@plamutplamut commented Jul 11, 2020

Copy link
Copy Markdown
Contributor

Fixes#151.

This PR makes sure that unknown fields are not silently ignored when inserting rows, but instead such errors are detected and raised loudly.

PR checklist

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

@googlebotgooglebot added the cla: yes This human has signed the Contributor License Agreement. label Jul 11, 2020
@plamut
plamut requested a review from shollymanJuly 11, 2020 12:51
@plamutplamut added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 13, 2020
@yoshi-kokoroyoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 13, 2020

@shollymanshollyman left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'm missing something here. The user controls whether they want this behavior in tabledata.insertall via ignoreUnknownValues, which is present in the insert_rows_json() method signature, but I don't see how that's plumbed to this helper.

Comment threadgoogle/cloud/bigquery/_helpers.py
Comment threadgoogle/cloud/bigquery/_helpers.py Outdated
@plamut

Copy link
Copy Markdown
ContributorAuthor

@shollyman Hmm, good point. I'll check how the helper can support the ignore_unknown_values parameter.

@plamut

Copy link
Copy Markdown
ContributorAuthor

Changed the helper to include missing fields and let the backend handle that.

The ignore_unknown_values argument to user-facing methods in the client is orthogonal to that, thus the helper signature didn't need a change (FWIW, the backend honors the argument, checked it).

@plamut

Copy link
Copy Markdown
ContributorAuthor

Just the coverage failure it seems, will cover the missed code path.

@tseaver

Copy link
Copy Markdown
Contributor

Assuming that the Python3 test failures in test_download_arrow_tabledata_list_unknown_field_type due to a pyarrow release, I've kicked off a merge.

@tseaver
tseaver merged commit 8fe7254 into googleapis:masterJul 30, 2020
@plamut

Copy link
Copy Markdown
ContributorAuthor

Indeed, and those tests were updated in https://github.com/googleapis/python-bigquery/pull/192/files

@plamut
plamut deleted the iss-151 branch July 31, 2020 08:40
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

cla: yesThis human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

client.insert_rows should fail when inserting non-existing fields

6 participants

@plamut@tseaver@tswast@shollyman@googlebot@yoshi-kokoro