Skip to content

BigQuery: Fix schema recognition of struct field types - #9001

Merged
tswast merged 2 commits into
googleapis:masterfrom
plamut:fix-struct_types
Aug 14, 2019
Merged

BigQuery: Fix schema recognition of struct field types#9001
tswast merged 2 commits into
googleapis:masterfrom
plamut:fix-struct_types

Conversation

@plamut

@plamutplamut commented Aug 9, 2019

Copy link
Copy Markdown
Contributor

A struct field can be referred to as "RECORD" or "STRUCT", and this commit assures that the to_api_repr() logic is correct. It also moves STRUCT_TYPES from _pandas_helpers to schema where it belongs more naturally IMO.

Discovered this while writing the "all types" system test, and tried to create a STRUCT field using the standard SQL type name "STRUCT".

How to test

  • Using the BigQuery client, try to create a table with a "STRUCT" column (using that particular type name):
    fromgoogle.cloudimportbigquerybq_client=bigquery.Client()
    schema= [
    bigquery.SchemaField(
    "person_struct_field",
    "STRUCT",
    fields=(
    bigquery.SchemaField("name", "STRING"),
    bigquery.SchemaField("age", "INT64"),
    )
    ),
    ]
    bq_table=bigquery.table.Table(
    table_ref="{}.{}.active_users".format(PROJECT_ID, DATASET_ID),
    schema=schema,
    )
    bq_client.create_table(bq_table)

Actual result (before the fix):
The server responds with an error:

Field <field_name> is type RECORD but has no schema

Expected result (after the fix):
The table is successfully created.

A struct field can be referred to as "RECORD" or "STRUCT", and this
commit assures that the to_api_repr() logic is correct.
@plamutplamut added the api: bigquery Issues related to the BigQuery API. label Aug 9, 2019
@plamut
plamut requested review from a team and tswastAugust 9, 2019 13:16
@googlebotgooglebot added the cla: yes This human has signed the Contributor License Agreement. label Aug 9, 2019
Comment threadbigquery/google/cloud/bigquery/schema.py Outdated
@plamut
plamut requested a review from tswastAugust 13, 2019 21:47
@tswast
tswast merged commit 66fe46e into googleapis:masterAug 14, 2019
emar-kar pushed a commit to MaxxleLLC/google-cloud-python that referenced this pull request Sep 18, 2019
* Fix schema recognition of struct field types
A struct field can be referred to as "RECORD" or "STRUCT", and this
commit assures that the to_api_repr() logic is correct.
* Mark STRUCT_TYPES as private in schema.py
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigqueryIssues related to the BigQuery API.cla: yesThis human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@plamut@tswast@googlebot