Uh oh!
There was an error while loading. Please reload this page.
feat(bigquery): add create_bqstorage_client param to to_dataframe and to_arrow - #9573
Merged
Conversation
shollyman
commented
Oct 31, 2019
Contributor
feedback during informal review: be clearer we're only using default settings when constructing the client on user behalf, and that to do more you'll still want to supply your own client. |
use_bqstorage_api param to to_dataframe and `…create_bqstorage_client param to to_dataframe and to_arrowtswast
marked this pull request as ready for review
November 1, 2019 22:41
plamut
reviewed
Nov 4, 2019
plamut
left a comment
Contributor
There was a problem hiding this comment.
Looks good, I only have a few minor comments.
Making it more clear that auto-creating a BQ storage client behind the scenes only uses the default settings would indeed be beneficial, as @shollyman pointed out.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
plamut
approved these changes
Nov 5, 2019
plamut
left a comment
Contributor
There was a problem hiding this comment.
LGTM.
The remaining remark is trivial, feel free to ignore, as the linter appears to not bother.
| download_public_data_sandbox.download_public_data_sandbox(client) | ||
| out, _ = capsys.readouterr() | ||
| out, err = capsys.readouterr() |
tswast
commented
Nov 5, 2019
ContributorAuthor
Per #9457 I should close the client transport if we create it in the method. |
create_bqstorage_client param to to_dataframe and to_arrowcreate_bqstorage_client param to to_dataframe and to_arrow… and `to_arrow` When the `create_bqstorage_client` parameter is set to `True`, the BigQuery client constructs a BigQuery Storage API client for you. This removes the need for boilerplate code to manually construct both clients explitly with the same credentials. Does this make the `bqstorage_client` parameter unnecessary? In most cases, yes, but there are a few cases where we'll want to continue using it. * When partner tools use `to_dataframe`, they should continue to use `bqstorage_client` so that they can set the correct amended user-agent strings. * When a developer needs to override the default API endpoint for the BQ Storage API, they'll need to manually supply a `bqstorage_client`.
create_bqstorage_client param to to_dataframe and to_arrowcreate_bqstorage_client param to to_dataframe and to_arrow
This was referenced Jan 29, 2020
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When the
create_bqstorage_clientparameter is set toTrue, the BigQueryclient constructs a BigQuery Storage API client for you. This removes
the need for boilerplate code to manually construct both clients
explicitly with the same credentials.
Does this make the
bqstorage_clientparameter unnecessary? In mostcases, yes, but there are a few cases where we'll want to continue using
it. Specifically, when partner tools use
to_dataframe, they shouldcontinue to use
bqstorage_clientso that they can set the correctamended user-agent strings.
bqstorage_clientis also needed for regionalAPI endpoints.
TODO
owns_bqstorage_clientto track ifbqstorage_clientneeds its transport closed.owns_bqstorage_clientis true.max_resultsis set.