Uh oh!
There was an error while loading. Please reload this page.
feat: use BigQuery Storage client by default - #55
Conversation
25572ab to
1bb7283Compareplamut
commented
Apr 7, 2020
I did some timings to compare fetching data with and without the BQ Storage client, results below. Spoiler: When using the BQ Storage API, performance improvements are enormous. QuerySource tables
Timings
|
fde9441 to
c01648dComparef080813 to
1aa0c62Compare1aa0c62 to
3f73a43Compare3f73a43 to
00bf584CompareUh oh!
There was an error while loading. Please reload this page.
12b2171 to
dbef14dCompare
shollyman
left a comment
There was a problem hiding this comment.
Semantic satiation seeing all the v1/v1beta references. Sorry it took me so long to get to this. Couple minor nits, with a couple more interesting bits (avro vs arrow, small result set, versioning for storage).
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
5b82c76 to
c715a74Compare* feat: add HOUR support for time partitioning interval
486309f to
6e35d09Compare
shollyman
left a comment
There was a problem hiding this comment.
Thanks for the slogging through this.
plamut
commented
May 15, 2020
Just a thought - since this is a pretty significant change, how about releasing all the other smaller changes and fixes that have accumulated since the last release separately, and release this one separately for an easier rollback, should that be necessary? |
shollyman
commented
May 15, 2020
Isolating this change to its own release seems prudent. |
Putting this on hold until the next release is made, we will ship it separately. |
plamut
commented
Jun 10, 2020
With the new release (1.25.0) now out, we can now merge this and release in the near future (possibly with miscellaneous related cleanup fixes). |
Closes#86.
Closes#84.
No issue yet, just a POC preview.The PR is now ready to be reviewed. It uses the BQ Storage API by default to fetch query results, and removes the "beta" label from that feature.
It's probably easier to review this PR commit by commit.
Key points: