Skip to content

Add partition support to BigQuery - #2058

Merged
tseaver merged 4 commits into
googleapis:masterfrom
dwmclary:add_partition_support
Aug 5, 2016
Merged

Add partition support to BigQuery#2058
tseaver merged 4 commits into
googleapis:masterfrom
dwmclary:add_partition_support

Conversation

@dwmclary

Copy link
Copy Markdown
Contributor

This adds the ability to create a partitioned table. It would be nice to expose a table.list_partitions() method, but currently there is not an API-based way of returning partitions. So, for now, it simply allows users to interact with the current partitioning API.

@googlebotgooglebot added the cla: yes This human has signed the Contributor License Agreement. label Aug 5, 2016
@dwmclarydwmclary changed the title Add partition supportAdd partition support to BigQueryAug 5, 2016
@tseavertseaver added the api: bigquery Issues related to the BigQuery API. label Aug 5, 2016
Comment threadgcloud/bigquery/table.py Outdated
"""
partitioned = None
if "timePartitioning" in self._properties:
partitioned = self._properties.get('timePartitioning').get('type')

This comment was marked as spam.

@tseaver

Copy link
Copy Markdown
Contributor

The docs for creating partitioned tables point to a query for listing partitions, Could we use that to implement `table.list_partitions'?

with self.assertRaises(ValueError):
table.partitioning_type = "HASH"
with self.assertRaises(ValueError):
table.partition_expiration = "NEVER"

This comment was marked as spam.

@tseaver

Copy link
Copy Markdown
Contributor

Thank you for the patch!

@tseavertseaver self-assigned this Aug 5, 2016
@dwmclary

Copy link
Copy Markdown
ContributorAuthor

I've got a method I didn't include for list_partitions that does generate and run that query. What I'm not sure about is the best way to fake a partitioned table for testing said method.

@tseaver

Copy link
Copy Markdown
Contributor

What I'm not sure about is the best way to fake a partitioned table for testing said method.

Assuming the method looks something like:

deflist_partitions(self):
query=self._client.query(
'SELECT partition_id from [%s.%s$__PARTITIONS_SUMMARY__]'%
(self.dataset_name, self.name))
query.run()
return [row[0] forrowinquery.rows]

I think I would just mock up the query method on the _Client mock to return an instance of a mocked _Query class: it would then have a run method which would populate its rows attribute.

@tseaver
tseaver merged commit a9f701a into googleapis:masterAug 5, 2016
@dhermesdhermes mentioned this pull request Sep 19, 2016
parthea added a commit that referenced this pull request Mar 6, 2026
Co-authored-by: Lingqing Gan <lingqing.gan@gmail.com>
parthea pushed a commit that referenced this pull request Apr 1, 2026
#2058)
* perf: avoid re-authenticating if credentials have already been fetched
* Update bigframes/_config/bigquery_options.py
* move lock to module
parthea pushed a commit that referenced this pull request Apr 1, 2026
* fix: allow bigframes.options.bigquery.credentials to be `None`
This is a partial revert of "perf: avoid re-authenticating if
credentials have already been fetched (#2058)", commit
913de1b31f3bb0b306846fddae5dcaff6be3cec4.
* add unit test
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigqueryIssues related to the BigQuery API.cla: yesThis human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@dwmclary@tseaver@googlebot