feat: add support for INTERVAL data type to `list_rows` by tswast · Pull Request #840 · googleapis/python-bigquery

tswast · 2021-07-29T22:12:30Z

Implementation of most other features for INTERVAL is blocked by backend changes.

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
Ensure the tests and linter pass
Code coverage does not decrease (if any source code was changed)
Appropriate docs were updated (if necessary)

Towards googleapis/google-cloud-python#15740 🦕

tswast · 2021-07-29T22:14:54Z

Some of these commits are from #829

jimfulton · 2021-07-30T12:12:13Z

+ minutes = time_sign * int(parsed.group("minutes"))
+ seconds = time_sign * int(parsed.group("seconds"))
+ fraction = parsed.group("fraction")
+ microseconds = time_sign * int(fraction.ljust(6, "0")) if fraction else 0


I can't tell from https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#interval_type if there are limits on any of these fields. 🤷

I agree. The docs don't really explain much about how this data format actually works. From what is documented and trial and error, I think these are the limits:
Min
years: -10000
months: -11 (I think?)
days: -3660000
hours: -87840000
minutes: -59 (I think?)
seconds: -59 (I think?)
microseconds: -999999 (I think?)
Max
years: 10000
months: 11 (I think?)
days: 3660000
hours: 87840000
minutes: 59 (I think?)
seconds: 59 (I think?)
microseconds: 999999 (I think?)
I don't think we'll need any client-side validation for this, but it does remind me that we should have some unit tests that exercise these limits.

jimfulton · 2021-07-30T12:14:59Z

+ BOOL = ScalarQueryParameterType("BOOL")
+ BOOLEAN = ScalarQueryParameterType("BOOL")
+ BIGDECIMAL = ScalarQueryParameterType("BIGNUMERIC")
+ BIGNUMERIC = ScalarQueryParameterType("BIGNUMERIC")
 BYTES = ScalarQueryParameterType("BYTES")
- INTEGER = ScalarQueryParameterType("INT64")
- INT64 = ScalarQueryParameterType("INT64")
+ DATE = ScalarQueryParameterType("DATE")
+ DATETIME = ScalarQueryParameterType("DATETIME")
+ DECIMAL = ScalarQueryParameterType("NUMERIC")
 FLOAT = ScalarQueryParameterType("FLOAT64")
 FLOAT64 = ScalarQueryParameterType("FLOAT64")
- NUMERIC = ScalarQueryParameterType("NUMERIC")
- BIGNUMERIC = ScalarQueryParameterType("BIGNUMERIC")
- DECIMAL = ScalarQueryParameterType("NUMERIC")
- BIGDECIMAL = ScalarQueryParameterType("BIGNUMERIC")
- BOOLEAN = ScalarQueryParameterType("BOOL")
- BOOL = ScalarQueryParameterType("BOOL")
 GEOGRAPHY = ScalarQueryParameterType("GEOGRAPHY")
- TIMESTAMP = ScalarQueryParameterType("TIMESTAMP")
- DATE = ScalarQueryParameterType("DATE")
+ INT64 = ScalarQueryParameterType("INT64")
+ INTEGER = ScalarQueryParameterType("INT64")
+ NUMERIC = ScalarQueryParameterType("NUMERIC")
+ STRING = ScalarQueryParameterType("STRING")
 TIME = ScalarQueryParameterType("TIME")
-DATETIME = ScalarQueryParameterType("DATETIME")
+TIMESTAMP = ScalarQueryParameterType("TIMESTAMP")


Did you intend to include INTERVAL?

I did but then reverted it because the query parameter support is incomplete. I could revert the alphabetization, but I figure we should do that at some point.

…-interval

tswast · 2021-08-05T21:47:49Z

I think this is ready, but I'll keep the Do Not Merge label, as I think #829 should go in first.

…-interval

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

…-interval

tswast · 2021-10-25T20:52:42Z

@plamut Did you have any additional thoughts on this? It'd be good to get in before v3. I added some proposed unit tests with trailing ., but kept the regex for separating seconds from microseconds due to rounding errors discovered with the divmod approach.

plamut

Went over this again, it looks fine (just one nit).

Good that we caught the issue in the divmod-based implementation, keeping the regex is the right way to go.

…-interval

* test: refactor `list_rows` tests and add test for scalars * WIP: INTERVAL support * feat: add support for INTERVAL data type to `list_rows` * fix relativedelta construction for non-microseconds * WIP: support INTERVAL query params * remove dead code * INTERVAL not supported in query parameters * revert query parameter changes * add validation error for interval * add unit tests for extreme intervals * add dateutil to intersphinx * use dictionary for intersphinx * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * add test case for trailing . * explicit none * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * truncate nanoseconds * use \d group for digits * use \d for consistency Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Peter Lamut <plamut@users.noreply.github.com>

tswast added 8 commits July 28, 2021 14:58

test: refactor list_rows tests and add test for scalars
5704f1e

WIP: INTERVAL support
5eb3794

feat: add support for INTERVAL data type to list_rows
9e54277

Merge remote-tracking branch 'upstream/master' into issue826-interval
a89c1c1

fix relativedelta construction for non-microseconds
60d9ca7

WIP: support INTERVAL query params
da6ef5b

remove dead code
b73a610

INTERVAL not supported in query parameters
52f2b7b

tswast requested review from a team and stephaniewang526 and removed request for a teamJuly 29, 2021 22:12

product-auto-labelBot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Jul 29, 2021

google-claBot added the cla: yes This human has signed the Contributor License Agreement. label Jul 29, 2021

Merge remote-tracking branch 'upstream/master' into issue826-interval
ca8872e

tswast added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Jul 29, 2021

tswast mentioned this pull request Mar 2, 2026
Support INTERVAL data type googleapis/google-cloud-python#15740
Open
4 tasks

jimfulton reviewed Jul 30, 2021
View reviewed changes

tswast added 4 commits August 3, 2021 11:10

Merge branch 'master' into issue826-interval
018a617

Merge remote-tracking branch 'upstream/master' into issue826-interval
2872d85

revert query parameter changes
31c3f92

Merge remote-tracking branch 'origin/issue826-interval' into issue826…
e3a3a6a
…-interval

plamut mentioned this pull request Aug 4, 2021
feat: support ScalarQueryParameterType for type_ argument in ScalarQueryParameter constructor #850
Merged
4 tasks

tswast added 3 commits August 5, 2021 16:07

Merge branch 'master' into issue826-interval
5f78311

add validation error for interval
bb03618

add unit tests for extreme intervals
f3711e7

tswast commented Aug 11, 2021
View reviewed changes

Comment threaddocs/conf.py Outdated

add dateutil to intersphinx
68035ba

tswast mentioned this pull request Sep 30, 2021
potential rounding error with use of divmodgoogleapis/python-db-dtypes-pandas#18
Closed

tswast added 2 commits September 30, 2021 13:32

add test case for trailing .
0497b19

Merge remote-tracking branch 'origin/issue826-interval' into issue826…
92f41b9
…-interval

tswast requested review from plamut and removed request for jimfulton September 30, 2021 18:38

explicit none
0318f54

tswast added the owlbot:run Add this label to trigger the Owlbot post processor. label Sep 30, 2021

gcf-owl-botBot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Sep 30, 2021

gcf-owl-botBotand others added 6 commits September 30, 2021 18:44

🦉 Updates from OwlBot
6b1f238
See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

truncate nanoseconds
54e47f7

Merge remote-tracking branch 'origin/issue826-interval' into issue826…
ced356b
…-interval

use \d group for digits
dcc8b57

Merge branch 'main' into issue826-interval
5c31fe2

Merge branch 'main' into issue826-interval
2091478

tswast added the owlbot:run Add this label to trigger the Owlbot post processor. label Oct 25, 2021

gcf-owl-botBot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Oct 25, 2021

Merge branch 'main' into issue826-interval
21a2975

plamut reviewed Oct 26, 2021
View reviewed changes

Comment threadgoogle/cloud/bigquery/_helpers.py Outdated

tswast added 3 commits October 26, 2021 10:09

Merge remote-tracking branch 'upstream/main' into issue826-interval
87b0d81

Merge remote-tracking branch 'origin/issue826-interval' into issue826…
7bd48be
…-interval

use \d for consistency
d62b950

tswast requested a review from plamut October 26, 2021 15:20

plamut approved these changes Oct 26, 2021
View reviewed changes

tswast added the automerge Merge the pull request once unit tests and other checks pass. label Oct 26, 2021

tswast merged commit e37380a into googleapis:mainOct 26, 2021

gcf-merge-on-greenBot removed the automerge Merge the pull request once unit tests and other checks pass. label Oct 26, 2021

tswast deleted the issue826-interval branch October 26, 2021 16:04

release-pleaseBot mentioned this pull request Jan 4, 2022
chore(main): release python-bigquery 1.27.1 #1097
Closed

Navigation Menu

Uh oh!

feat: add support for INTERVAL data type to `list_rows` - #840

feat: add support for INTERVAL data type to `list_rows`#840
tswast merged 39 commits into
googleapis:mainfrom
tswast:issue826-interval

tswast commented Jul 29, 2021

Uh oh!

tswast commented Jul 29, 2021

Uh oh!

Uh oh!

jimfultonJul 30, 2021

Uh oh!

tswastAug 5, 2021

Uh oh!

jimfultonJul 30, 2021

Uh oh!

tswastAug 3, 2021

Uh oh!

Uh oh!

Uh oh!

tswast commented Aug 5, 2021

Uh oh!

Uh oh!

tswast commented Oct 25, 2021

Uh oh!

plamut left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

tswast commented Jul 29, 2021

Uh oh!

tswast commented Jul 29, 2021

Uh oh!

Uh oh!

jimfultonJul 30, 2021

Choose a reason for hiding this comment

Uh oh!

tswastAug 5, 2021

Choose a reason for hiding this comment

Uh oh!

jimfultonJul 30, 2021

Choose a reason for hiding this comment

Uh oh!

tswastAug 3, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tswast commented Aug 5, 2021

Uh oh!

Uh oh!

tswast commented Oct 25, 2021

Uh oh!

plamut left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants