Skip to content

BigQuery: Add routines support - #5591

Merged
shollyman merged 18 commits into
googleapis:masterfrom
shollyman:routines
Jul 12, 2019
Merged

BigQuery: Add routines support #5591
shollyman merged 18 commits into
googleapis:masterfrom
shollyman:routines

Conversation

@shollyman

@shollymanshollyman commented Jun 28, 2019

Copy link
Copy Markdown
Contributor

This adds java support for bigquery routines, which enables API access to a persistent functions and (soon) stored procedures. Routines exist inside of BigQuery datasets, just like tables and BQML models.

UDF documentation:
https://cloud.google.com/bigquery/docs/reference/standard-sql/user-defined-functions
Routines REST documentation:
https://cloud.google.com/bigquery/docs/reference/rest/v2/routines

This PR adds:
CRUD operations for manipulating Routine resources. It models the same structure used for Tables and Models. There's a RoutineId class for the qualified reference, a RoutineInfo for metadata and a Routine class for binding to the service operations like create/update/delete. Submessages like arguments (RoutineArgument) are also implemented as classes.

StandardSQLDataType,StandardSQLField,StandardSQLStructType veneers for communicating type information using standard sql typing. These are used for communicating datatype information for inputs/outputs in routines.

Augments query stats: In addition to using the API directly, you can also manipulate routines using the query language (e.g. CREATE FUNCTION ... AS ...). Statistics provided as part of the query job include the statement type, and when using appropriate DDL queries the DDL target routine.

There's some other cleanup around typos/docs, as well as filling in missing things like statements in the StatementType enum.

@shollyman
shollyman requested a review from a teamJune 28, 2019 18:38
@googlebotgooglebot added the cla: yes This human has signed the Contributor License Agreement. label Jun 28, 2019
@shollymanshollyman added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Jun 28, 2019
@shollyman

Copy link
Copy Markdown
ContributorAuthor

Failure is due to unrelated flaky test in BQ, sent #5592 to address.

@codecov

codecovBot commented Jul 8, 2019

Copy link
Copy Markdown

Codecov Report

Merging #5591 into master will increase coverage by 0.37%.
The diff coverage is 53.65%.

Impacted file tree graph

@@ Coverage Diff @@## master #5591 +/- ##
============================================
+ Coverage 46.71% 47.09% +0.37% - Complexity 24647 25078 +431 
============================================
Files 2351 2389 +38 Lines 256185 259732 +3547 Branches 29333 29397 +64 ============================================
+ Hits 119688 122317 +2629 - Misses 127575 128454 +879 - Partials 8922 8961 +39
Impacted FilesCoverage ΔComplexity Δ
.../com/google/cloud/bigquery/spi/v2/BigQueryRpc.java77.77% <ø> (ø)0 <0> (ø)⬇️
...e/cloud/bigquery/testing/RemoteBigQueryHelper.java61.4% <0%> (-1.1%)6 <0> (ø)
...n/java/com/google/cloud/bigquery/BigQueryImpl.java66.78% <0%> (-13.22%)57 <0> (ø)
.../google/cloud/bigquery/spi/v2/HttpBigQueryRpc.java5.3% <0%> (-0.8%)2 <0> (ø)
.../java/com/google/cloud/bigquery/JobStatistics.java93.91% <100%> (+0.34%)13 <0> (ø)⬇️
...in/java/com/google/cloud/bigquery/RoutineInfo.java66.95% <66.95%> (ø)20 <20> (?)
...com/google/cloud/bigquery/StandardSQLDataType.java75% <75%> (ø)7 <7> (?)
...main/java/com/google/cloud/bigquery/RoutineId.java76.92% <76.92%> (ø)13 <13> (?)
.../main/java/com/google/cloud/bigquery/BigQuery.java78.75% <80.76%> (+0.31%)0 <0> (ø)⬇️
...c/main/java/com/google/cloud/bigquery/Routine.java82.35% <82.35%> (ø)11 <11> (?)
... and 46 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d67dc01...d9a549d. Read the comment docs.

@shollymanshollyman removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Jul 9, 2019
@shollyman
shollyman requested a review from tswastJuly 9, 2019 17:35
}

/** Returns an option to specify the maximum number of models returned per page. */
public static RoutineListOption pageSize(long pageSize) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be package private? (Same with other public methods in package private classes)

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably reasonable, we extend Option but don't appear to extend each of the various typed options beyond that. Filed https://github.com/googleapis/google-cloud-java/issues/5728 to followup on visibility more generally.

@chingor13chingor13 changed the title bigquery: add routines support BigQuery: Add routines support Jul 11, 2019
@shollyman
shollyman merged commit 7d4f53f into googleapis:masterJul 12, 2019
@shollyman
shollyman deleted the routines branch July 12, 2019 16:01
@JesseLovelaceJesseLovelace mentioned this pull request Jul 17, 2019
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yesThis human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@shollyman@sduskis@tswast@googlebot