Skip to content

Added support for schema auto-detection feature in the job method `Lo… - #3648

Merged
dhermes merged 4 commits into
googleapis:masterfrom
WillianFuks:add-bigquery-autodetect-schema-LoadTableFromStorageJob
Jul 28, 2017
Merged

Added support for schema auto-detection feature in the job method `Lo…#3648
dhermes merged 4 commits into
googleapis:masterfrom
WillianFuks:add-bigquery-autodetect-schema-LoadTableFromStorageJob

Conversation

@WillianFuks

@WillianFuksWillianFuks commented Jul 21, 2017•
edited
Loading

Copy link
Copy Markdown
Contributor

Hello,

I did some implementation adding support for the feature "schema auto-detection" for BigQuery in the job method LoadTableFromStorageJob.

Not sure if got it right, please let me know what you think and I can fix whatever issues you may find.

@googlebotgooglebot added the cla: yes This human has signed the Contributor License Agreement. label Jul 21, 2017
@dhermesdhermes added the api: bigquery Issues related to the BigQuery API. label Jul 21, 2017
@dhermes

dhermes commented Jul 21, 2017

Copy link
Copy Markdown
Contributor

I'll defer to @tswast or @jonparrott on review here. (In particular, review of the surface vs. review of the code itself.)

@theacodes

theacodes commented Jul 21, 2017

Copy link
Copy Markdown
Contributor

@tswast is out-of-office. This surface seems reasonable to me.

tswast
tswast reviewed Jul 24, 2017
View reviewed changes
Comment threadbigquery/google/cloud/bigquery/job.py Outdated
https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load.allowQuotedNewlines
"""

autodetect_schema = AutoDetectSchema('autodetect_schema', bool)

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

@tswast

tswast commented Jul 26, 2017

Copy link
Copy Markdown
Contributor

@dhermes could you do a review pass, too, and merge if it looks okay to you?

@tseavertseaver self-assigned this Jul 26, 2017
@dhermes

dhermes commented Jul 26, 2017

Copy link
Copy Markdown
Contributor

Working on it now

@dhermes

dhermes commented Jul 26, 2017

Copy link
Copy Markdown
Contributor

@tseaver I noticed you assigned yourself, don't want to cross paths here but I've been actually in a local checkout of the code for about 15 minutes.

@tseavertseaver assigned dhermes and unassigned tseaverJul 26, 2017
@tseaver

tseaver commented Jul 26, 2017

Copy link
Copy Markdown
Contributor

@dhermes Go ahead!

@googlebot

googlebot commented Jul 26, 2017

Copy link
Copy Markdown

So there's good news and bad news.

👍 The good news is that everyone that needs to sign a CLA (the pull request submitter and all commit authors) have done so. Everything is all good there.

😕 The bad news is that it appears that one or more commits were authored by someone other than the pull request submitter. We need to confirm that they're okay with their commits being contributed to this project. Please have them confirm that here in the pull request.

Note to project maintainer: This is a terminal state, meaning the cla/google commit status will not change from this state. It's up to you to confirm consent of the commit author(s) and merge this pull request when appropriate.

@googlebotgooglebot added cla: no This human has *not* signed the Contributor License Agreement. and removed cla: yes This human has signed the Contributor License Agreement. labels Jul 26, 2017
@dhermes

dhermes commented Jul 26, 2017

Copy link
Copy Markdown
Contributor

@WillianFuks I just sent a commit into this PR with a few style tweaks (thanks for adhering to the existing style BTW).

I did have a question about your expectation of the _configuration value in the schema@property, but I'll ask it on the code.

dhermes
dhermes reviewed Jul 26, 2017
View reviewed changes
Comment threadbigquery/google/cloud/bigquery/job.py Outdated
else:
if not all(isinstance(field, SchemaField) for field in value):
raise ValueError('Schema items must be fields')
if self._configuration is not None and self.autodetect:

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

@WillianFuks

WillianFuks commented Jul 28, 2017

Copy link
Copy Markdown
ContributorAuthor

Hi @dhermes , just to confirm, is everything ok in this PR? I saw this message from googlebot saying there's an issue with CLA, I wonder if I need to do something related.

Thanks :)!

@dhermes

dhermes commented Jul 28, 2017

Copy link
Copy Markdown
Contributor

@WillianFuks The CLA bot gets confused when there are commits from multiple authors, sorry for that.

As for "is everything OK", I seem to recall that I was waiting on your for response on something, but I can't find any comment to that effect.

@dhermes

dhermes commented Jul 28, 2017

Copy link
Copy Markdown
Contributor

I think the issue may just be the failed docs build. I'll rebase and hopefully it'll have been fixed by #3683

WillianFuksand others added 4 commits July 28, 2017 09:59
@WillianFuks@dhermes
Added support for schema auto-detection feature in the job method `Lo…
1e5e03f
…adTableFromStorageJob`
@WillianFuks@dhermes
Changed property name autodetect_schema to autodetect to follow b…
8a851e3
…ackend API standard.
@dhermes
Style tweaks (for code review in#3648).
e68512e
Most importantly, updating the `getattr(self, '_configuration', None)`
to just access normally (the `schema` `@property` was being called
in the constructor **before** the `_configuration` attribute was
set).
@dhermes
Removing configuration check.
88007ee
@dhermes
dhermes merged commit b94a326 into googleapis:masterJul 28, 2017
@WillianFuks
WillianFuks deleted the add-bigquery-autodetect-schema-LoadTableFromStorageJob branch July 29, 2017 01:29
@tswasttswast mentioned this pull request Jul 30, 2017
Closed
landrito pushed a commit to landrito/google-cloud-python that referenced this pull request Aug 21, 2017
@WillianFuks@landrito
Added support for schema auto-detection feature in `LoadTableFromStor…
d45612a
…ageJob` (googleapis#3648)
landrito pushed a commit to landrito/google-cloud-python that referenced this pull request Aug 22, 2017
@WillianFuks@landrito
Added support for schema auto-detection feature in `LoadTableFromStor…
8d47e80
…ageJob` (googleapis#3648)
landrito pushed a commit to landrito/google-cloud-python that referenced this pull request Aug 22, 2017
@WillianFuks@landrito
Added support for schema auto-detection feature in `LoadTableFromStor…
bd916bc
…ageJob` (googleapis#3648)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tswasttswasttswast approved these changes
+1 more reviewer
@dhermesdhermesdhermes left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

api: bigqueryIssues related to the BigQuery API.cla: noThis human has *not* signed the Contributor License Agreement.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@WillianFuks@dhermes@theacodes@tswast@tseaver@googlebot