Add support for query parameters by tseaver · Pull Request #2776 · googleapis/google-cloud-python

tseaver · 2016-11-29T21:24:04Z

Holds name, type, and value for scalar query parameters, and handles marshalling them to / from JSON representation mandated by the BigQuery API. Toward #2551.

Holds name, type, and value for array query parameters, and handles marshalling them to / from JSON representation mandated by the BigQuery API. Toward #2551.

Holds name, types, and values for Struct query parameters, and handles marshalling them to / from JSON representation mandated by the BigQuery API. Toward #2551.

It is just a list comprehension, really.

Sign in to view

 from google.cloud.bigquery.dataset import Dataset
 from google.cloud.bigquery.schema import SchemaField
 from google.cloud.bigquery.table import Table
+from google.cloud.bigquery._helpers import ArrayQueryParameter


Sign in to view

+ """Named / positional query parameters for scalar values.
+
+ :type name: str or None
+ :param name: Parameter name, used via `@foo` syntax. If None, the


Sign in to view

+ @classmethod
+ def positional(cls, type_, value):
+ """Factory for positional paramters.
+


Sign in to view

+ :param value: the scalar parameter value.
+
+ :rtype: :class:`ScalarQueryParameter`
+ :returns: instance w/o name


Sign in to view

+ """Factory for positional paramters.
+
+ :type sub_parms: tuple of :class:`ScalarQueryParameter`
+ :param sub_parms:s the sub-parameters for the struct


Sign in to view

+
+ @staticmethod
+ def _get_target_class():
+ from google.cloud.bigquery._helpers import AbstractQueryParameter


Sign in to view

+ """Named / positional query parameters for scalar values.
+
+ :type name: str or None
+ :param name: Parameter name, used via `@foo` syntax. If None, the


Sign in to view

+class ScalarQueryParameter(object):
+ """Named / positional query parameters for scalar values.
+
+ :type name: str or None


Sign in to view

+
+ :type name: str or None
+ :param name: Parameter name, used via `@foo` syntax. If None, the
+ paramter can only be addressed via position (`?`).


Sign in to view

+ """
+ name = resource.get('name')
+ type_ = resource['parameterType']['type']
+ value = resource['parameterValue']['value']


Sign in to view

+ :class:`datetime.date`.
+ :param value: the scalar parameter value.
+ """
+ def __init__(self, name, type_, value):


Sign in to view

+ query_parameter.to_api_repr()
+ for query_parameter in self._query_parameters
+ ]
+ if self._query_parameters[0].name is None:


Sign in to view

 else:
 self.assertIsNone(job.maximum_bytes_billed)

+ def _verifyUDFResources(self, job, config):


Sign in to view

+ def test_begin_w_named_query_parameter(self):
+ from google.cloud.bigquery._helpers import ScalarQueryParameter
+ query_parameters = [ScalarQueryParameter('foo', 'INT64', 123)]
+ PATH = 'projects/%s/jobs' % self.PROJECT


Sign in to view

- if token is None:
- break
- jobs, token = client.list_jobs(page_token=token) # API request
+ job_iterator = client.list_jobs() # API request


Sign in to view


+@snippet
+def client_run_sync_query_w_param(client, _):
+ """Run a synchronous query using a query parameter"""


Addresses: #2776 (comment)

Addresses: #2776 (comment).

Addresses: #2776 (comment)

Sign in to view

+
+ :type type_: str
+ :param type_: name of parameter type. One of `'STRING'`, `'INT64'`,
+ `'FLOAT64'`, `'BOOLEAN'`, `'TIMESTAMP'`, or `'DATE'`.


The back-end raises an exception when passed the 'BOOLEAN' type to define a query parameter. Addresses: #2776 (comment)

Sign in to view

- {'name': name, 'type': self.struct_types[name]}
- for name in self._order
+ {'name': key, 'type': value}
+ for key, value in self.struct_types.items()


Sign in to view

+ the class).
+
+ :type value: list of instances of classes derived from
+ :class:`AbstractQueryParameter`.


Addresses: #2776 (comment).

dhermes

LGTM, though I don't understand why you'd have the constructor and the API rep so far apart, especially if no human is every going to use the constructor

tseaver · 2016-12-02T22:50:26Z

especially if no human is every going to use the constructor

I do expect humans to use the constructor, e.g.:

fromgoogle.cloud.bigqueryimportClientfromgoogle.cloud.bigqueryimportScalarQueryParameterfromgoogle.cloud.bigqueryimportStructQueryParameterbirth_state=ScalarQueryParameter('birth_state', 'STRING', 'VA')
death_state=ScalarQueryParameter('death_state', 'STRING', 'MS')
birth_death=StructQueryParameter(
'birth_death', birth_state, death_state)
client=Client()
query=client.run_sync_query(
'SELECT * FROM mortality WHERE birth_death = @birth_death')

* Add 'ScalarQueryParameter' class. Holds name, type, and value for scalar query parameters, and handles marshalling them to / from JSON representation mandated by the BigQuery API. * Factor out 'AbstractQueryParameter. * Add 'ArrayQueryParameter' class. Holds name, type, and value for array query parameters, and handles marshalling them to / from JSON representation mandated by the BigQuery API. * Add 'StructQueryParameter' class. Holds name, types, and values for Struct query parameters, and handles marshalling them to / from JSON representation mandated by the BigQuery API. * Add 'QueryParametersProperty' descriptor class. * Add 'query_parameters' property to 'QueryResults' and 'QueryJob'. * Plumb 'udf_resources'/'query_parameters' through client query factories. * Expose concrete query parameter classes as package APIs. Closesgoogleapis#2551.

avitzavi · 2018-10-31T17:36:18Z

does this mean we can use @parameters in the BigQuery UI ?? I cant seem to figure out how to declare them correctly.

tseaver · 2018-10-31T18:10:06Z

@avitzavi I'm not sure what you are asking. Does the example in the docs help?

avitzavi · 2018-10-31T18:38:55Z

I’m trying to use variables like I do in sql server: “Declare @variable INT = 10” Can I do this in the BigQuery UI?

…

On Oct 31, 2018, at 2:25 PM, Tres Seaver ***@***.***> wrote: @avitzavi I'm not sure what you are asking. Does the example in the docs help? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

tseaver · 2018-10-31T21:30:08Z

The @parameter placeholders added by this PR are used for passing in values from the client to be substituted into a query: they aren't meant to be used that way.

Also, I don't know what you mean by "BigQuery UI" -- is that the https://console.cloud.google.com/ interface?

avitzavi · 2018-10-31T22:10:05Z

Yes it is the console. The Declare/Set methods from SQL are very valuable for me as a data analyst. Seems like a really big missing feature... I create many reports (based off scheduled queries) and not having these parameters makes it much more challenging to do. Especially given the Console's lack of basic editor features like find/replace.

…

Oct 31, 2018, at 5:30 PM, Tres Seaver ***@***.***> wrote: The @parameter placeholders added by this PR are used for passing in values from the client to be substituted into a query: they aren't meant to be used that way. Also, I don't know what you mean by "BigQuery UI" -- is that the https://console.cloud.google.com/ interface? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

tseaver · 2018-10-31T22:34:11Z

@avitzavi This repository is solely about the Python client libraries which wrap the various Google Cloud APIs. Please check out the BigQuery support page for ways to get support for the back-end API itself.

tseaver added 10 commits November 28, 2016 16:50

Add 'ScalarQueryParameter' class.
128db62
Holds name, type, and value for scalar query parameters, and handles marshalling them to / from JSON representation mandated by the BigQuery API. Toward #2551.

Factor out 'AbstractQueryParameter.
3421d7a

Add 'ArrayQueryParameter' class.
57c61c1
Holds name, type, and value for array query parameters, and handles marshalling them to / from JSON representation mandated by the BigQuery API. Toward #2551.

Add 'StructQueryParameter' class.
6650c59
Holds name, types, and values for Struct query parameters, and handles marshalling them to / from JSON representation mandated by the BigQuery API. Toward #2551.

Remove '_build_udf_resources' helper.
fcff570
It is just a list comprehension, really.

Add 'QueryParametersProperty' descriptor class.
2e06228

Add 'query_parameters' property to 'QueryResults' and 'QueryJob'.
968582f

Plumb 'udf_resources'/'query_parameters' through client query factories.
52b25ec

Expose concrete query parameter classes as package APIs.
9089907

Add a snippet documenting use of query parameters.
9fd7f76

tseaver added the api: bigquery Issues related to the BigQuery API. label Nov 29, 2016

tseaver assigned daspecster and dhermes Nov 29, 2016

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Nov 29, 2016

daspecster reviewed Nov 29, 2016
View reviewed changes

Address code review.
1df82a0

dhermes reviewed Nov 29, 2016
View reviewed changes

daspecster added the PR: Needs to be squashed label Nov 29, 2016

tseaver added 5 commits November 30, 2016 10:58

Document expected types for QueryParametersProperty setter/getter.
6191ca6
Addresses: #2776 (comment)

Use 'collections.OrderedDict' vs. hand-rolling order.
89f5a58
Addresses: #2776 (comment).

PEP8.
a4ff964
Addresses: #2776 (comment).

Correct notation of where API requests occur.
d4d451e
Addresses: #2776 (comment)

s/parm/param/.
0f61970
Addresses: #2776 (comment)

tseaver mentioned this pull request Nov 30, 2016
BigQuery: Add support for query parameters #2551
Closed

blowmage reviewed Dec 1, 2016
View reviewed changes

tseaver mentioned this pull request Dec 1, 2016
Add support for standard SQL 'BOOL' type. #2789
Merged

Update docstring to show standard-SQL 'BOOL' type.
524baf1
The back-end raises an exception when passed the 'BOOLEAN' type to define a query parameter. Addresses: #2776 (comment)

danoscarmike mentioned this pull request Dec 1, 2016
bigquery: support SQL parameters googleapis/google-cloud-node#1787
Closed

dhermes reviewed Dec 2, 2016
View reviewed changes

tseaver added 3 commits December 2, 2016 14:41

Indentation fix.
4b2cb13

Preserve OrderedDict for struct_types.
99bdcd1
Addresses: #2776 (comment).

Normalize '%' formatting.
132741a
Addresses: #2776 (comment).

dhermes approved these changes Dec 2, 2016
View reviewed changes

tseaver merged commit 75c4115 into googleapis:masterDec 2, 2016

tseaver deleted the 2551-bigquery-query_parameters branch December 2, 2016 22:52

tseaver removed the PR: Needs to be squashed label Dec 2, 2016

dhermes mentioned this pull request Dec 5, 2016
Cutting new releases for BigQuery, Language and Core. #2815
Merged

tswast mentioned this pull request Dec 5, 2016
BigQuery samples using query parameters. GoogleCloudPlatform/python-docs-samples#699
Merged

theacodes unassigned daspecster and dhermes Sep 28, 2018

Uh oh!

Conversation

tseaver commented Nov 29, 2016

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

dhermes left a comment

Choose a reason for hiding this comment

Uh oh!

tseaver commented Dec 2, 2016

Uh oh!

avitzavi commented Oct 31, 2018

Uh oh!

tseaver commented Oct 31, 2018