chore: sync v3 with main by tswast · Pull Request #1004 · googleapis/python-bigquery

tswast · 2021-10-06T15:35:52Z

Thought: maybe if we merge this as a merge commit, we can avoid some conflicts the next time we sync?

* chore: protect v3.x.x branch In preparation for breaking changes. * force pattern to be a string * simplify branch name

…#815) That warning should only be used when BQ Storage client is explicitly passed in to RowIterator methods when max_results value is also set.

…ields for BQML. (#817) PiperOrigin-RevId: 387137741 Source-Link: googleapis/googleapis@8962c92 Source-Link: googleapis/googleapis-gen@102f1b4

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

…om_dataframe` (#832)

#795)

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

These two lines bring the rules on this repo in line with the defaults: https://github.com/googleapis/repo-automation-bots/blob/63c858e539e1f4d9bb8ea66e12f9c0a0de5fef55/packages/sync-repo-settings/src/required-checks.json#L40-L50

The backend API has a timeout of 4 minutes, so the default of 2 minutes was not allowing for any retries to happen in some cases. Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly: - [ ] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-bigquery/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [ ] Ensure the tests and linter pass - [ ] Code coverage does not decrease (if any source code was changed) - [ ] Appropriate docs were updated (if necessary) Fixes#853 🦕

Closes#857.

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Tim Swast <swast@google.com>

[![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [google-cloud-bigquery](https://togithub.com/googleapis/python-bigquery) | `==2.23.2` -> `==2.23.3` | [![age](https://badges.renovateapi.com/packages/pypi/google-cloud-bigquery/2.23.3/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/google-cloud-bigquery/2.23.3/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/google-cloud-bigquery/2.23.3/compatibility-slim/2.23.2)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/google-cloud-bigquery/2.23.3/confidence-slim/2.23.2)](https://docs.renovatebot.com/merge-confidence/) | *** ### Release Notes <details> <summary>googleapis/python-bigquery</summary> ### [`v2.23.3`](https://togithub.com/googleapis/python-bigquery/blob/master/CHANGELOG.md#2233-httpswwwgithubcomgoogleapispython-bigquerycomparev2232v2233-2021-08-06) [Compare Source](https://togithub.com/googleapis/python-bigquery/compare/v2.23.2...v2.23.3) </details> *** ### Configuration 📅 **Schedule**: At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. *** * \[ ] If you want to rebase/retry this PR, check this box. *** This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/python-bigquery).

* feat: add support for transaction statistics * Hoist transaction_info into base job class * Add versionadded directive to new property and class * Include new class in docs reference

#863) [![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [google-cloud-bigquery-storage](https://togithub.com/googleapis/python-bigquery-storage) | `==2.6.2` -> `==2.6.3` | [![age](https://badges.renovateapi.com/packages/pypi/google-cloud-bigquery-storage/2.6.3/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/google-cloud-bigquery-storage/2.6.3/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/google-cloud-bigquery-storage/2.6.3/compatibility-slim/2.6.2)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/google-cloud-bigquery-storage/2.6.3/confidence-slim/2.6.2)](https://docs.renovatebot.com/merge-confidence/) | *** ### Release Notes <details> <summary>googleapis/python-bigquery-storage</summary> ### [`v2.6.3`](https://togithub.com/googleapis/python-bigquery-storage/blob/master/CHANGELOG.md#263-httpswwwgithubcomgoogleapispython-bigquery-storagecomparev262v263-2021-08-06) [Compare Source](https://togithub.com/googleapis/python-bigquery-storage/compare/v2.6.2...v2.6.3) </details> *** ### Configuration 📅 **Schedule**: At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. *** * \[x] If you want to rebase/retry this PR, check this box. *** This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/python-bigquery).

Source-Link: googleapis/synthtool@6252f2c Post-Processor: gcr.io/repo-automation-bots/owlbot-python:latest@sha256:50e35228649c47b6ca82aa0be3ff9eb2afce51c82b66c4a03fe4afeb5ff6c0fc Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

* feat: make the same Table instances equal to each other * Table equality should ignore metadata differences * Compare instances through tableReference property * Make Table instances hashable * Make Table* classes interchangeable If these classes reference the same table, they are now considered equal.

…larQueryParameter` constructor (#850) Follow-up to https://github.com/googleapis/python-bigquery/pull/840/files#r679880582 Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly: - [ ] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-bigquery/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [ ] Ensure the tests and linter pass - [ ] Code coverage does not decrease (if any source code was changed) - [ ] Appropriate docs were updated (if necessary)

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly: - [x] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-bigquery/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [x] Ensure the tests and linter pass - [x] Code coverage does not decrease (if any source code was changed) - [x] Appropriate docs were updated (if necessary) Fixes#539 🦕 Previously, we only retried failed API requests. Now, we retry failed jobs (according to the predicate of the `Retry` object passed to `job.result()`).

Co-authored-by: Tim Swast <swast@google.com> Co-authored-by: Tres Seaver <tseaver@palladion.com>

tswast · 2021-10-07T18:23:46Z

Samples 3.10 failure is expected. I might want to delete that job for now.

tseaver · 2021-10-07T18:41:47Z

@tswast I think the change to the samples image is forthcoming soon.

tswast · 2021-10-07T18:54:41Z

I think the change to the samples image is forthcoming soon.

It's still going to fail for BQ, because we don't have wheels for many of our compiled dependencies. e.g. pandas, numpy, pyarrow.

)

@busunkim96

chore: fix docstring for first attribute of protos committer: @busunkim96 PiperOrigin-RevId: 401271153 Source-Link: googleapis/googleapis@787f8c9 Source-Link: googleapis/googleapis-gen@81decff Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiODFkZWNmZmU5ZmM3MjM5NmE4MTUzZTc1NmQxZDY3YTZlZWNmZDYyMCJ9 * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Tim Swast <swast@google.com>

_Thank_ you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly: - [ ] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-bigquery/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [ ] Ensure the tests and linter pass - [ ] Code coverage does not decrease (if any source code was changed) - [ ] Appropriate docs were updated (if necessary) Fixes#995 🦕

tswast · 2021-10-12T17:59:12Z

We'll want to wait for allowing merge commits before we merge this: #1018

This should prevent conflicts when we sync v3 with main. Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly: - [ ] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-bigquery/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [ ] Ensure the tests and linter pass - [ ] Code coverage does not decrease (if any source code was changed) - [ ] Appropriate docs were updated (if necessary)

* feat: allow queryJob.result() to be called on a dryRun * update to return EmptyRowIterator

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly: - [ ] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-bigquery/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [ ] Ensure the tests and linter pass - [ ] Code coverage does not decrease (if any source code was changed) - [ ] Appropriate docs were updated (if necessary) Issue discovered while investigating what properties are needed in #967

tswast · 2021-10-15T17:35:27Z

CC @plamut , I think for future syncs from main to v3 we should use merge commits. That seems to make it easier as I add commits from main to this PR.

…1023) While implementing Sessions, I noticed several missing classes in the reference docs. Using "automodule" since experience has shown we often forget to add new classes to `docs/reference.rst`.

plamut · 2021-10-16T10:04:44Z

@tswast That's actually a very sensible idea, as I had the deja vu feeling when syncing the v3 branch for the second time. Some of the conflicts felt very familiar, although at the time I did not remember all the details anymore.

plamut

It seems fine, apart from the intersphinx links that should be retained.

The unit tests still produce no warnings, and the skipif decorators in tests do not check for dependencies that have been made required in v3 (BQ storage, pyarrow...) - these are the two main things that can sneak in without a merge conflict, as far as I can remember.

plamut · 2021-10-16T10:17:27Z

- "proto-plus": ("https://proto-plus-python.readthedocs.io/en/latest/", None),
- "protobuf": ("https://googleapis.dev/python/protobuf/latest/", None),


IIRC these were added these ack when we re-added legacy proto-based types to v3. We should thus retain them.

plamut · 2021-10-16T10:18:30Z

-# Remove unneeded intersphinx links, the library does not use any proto-generated code.
-s.replace("docs/conf.py", r'\s+"(proto-plus|protobuf)":.*$', "")


Related to the comment above, we actually want to include proto* intersphinx links.

This line was actually removing them from the template. Deleting this line, we retain the ones from https://github.com/googleapis/synthtool/blob/694118b039b09551fb5d445fceb361a7dbb06400/synthtool/gcp/templates/python_library/docs/conf.py.j2#L369-L370 and don't need to add them in the intersphinx_dependencies dictionary.

Mailed #1035 to do this separately rather than as a merge from main.

plamut · 2021-10-16T10:34:12Z

Added the do not merge label as an explicit reminder to use a merge commit when merging this (do NOT squash). Hopefully the person who eventually merges this will notice this comment when seeking the reason why the label was added in the first place. 🙃

#1030) Source-Link: googleapis/synthtool@7fd61f8 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:4ee57a76a176ede9087c14330c625a71553cf9c72828b2c0ca12f5338171ba60 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

plamut · 2021-10-28T06:58:30Z

Used a merged commit, this should get rid of the ever-growing list of merge conflicts every time we sync v3 with main. We should thus use merge commits to every future sync PRs, too.

tswastand others added 30 commits July 27, 2021 13:04

chore: protect v3.x.x branch (#816)
3b70891
* chore: protect v3.x.x branch In preparation for breaking changes. * force pattern to be a string * simplify branch name

fix: no longer raise a warning into_dataframeifmax_resultsset (…
3c1be14
…#815) That warning should only be used when BQ Storage client is explicitly passed in to RowIterator methods when max_results value is also set.

feat: Update proto definitions for bigquery/v2 to support new proto f…
fe7a902
…ields for BQML. (#817) PiperOrigin-RevId: 387137741 Source-Link: googleapis/googleapis@8962c92 Source-Link: googleapis/googleapis-gen@102f1b4

chore: release 2.23.0 (#819)
02bbdae
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

chore(deps): update dependency google-cloud-bigquery to v2.23.0 (#820)
42b66d3

fix:insert_rows()accepts float column values as strings again (#824)
d9378af

chore: release 2.23.1 (#825)
a505440
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

chore: add second protection rule for v3 branch (#828)
c541c69

chore(deps): update dependency google-cloud-bigquery to v2.23.1 (#827)
48e8a35

test: retry getting rows after streaming them in `test_insert_rows_fr…
d8c25ac
…om_dataframe` (#832)

chore(deps): update dependency pyarrow to v5 (#834)
8149d9e

chore(deps): update dependency google-cloud-bigquery-storage to v2.6.2 (
b9349ad
#795)

deps: expand pyarrow pins to support 5.x releases (#833)
80e3a61

chore: release 2.23.2 (#835)
40ef77f
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

chore(deps): update dependency google-auth-oauthlib to v0.4.5 (#839)
55687b8

chore(deps): update dependency google-cloud-bigquery to v2.23.2 (#838)
85ce81c

chore(deps): update dependency google-cloud-testutils to v1 (#845)
20df24b

chore: add api-bigquery as a samples owner (#852)
cf0b0d8

process: add yoshi-python to samples CODEOWNERS (#858)
e2cbcaa
Closes#857.

chore: release 2.23.3 (#860)
9694a4d
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Tim Swast <swast@google.com>

feat: add support for transaction statistics (#849)
7f7b1a8
* feat: add support for transaction statistics * Hoist transaction_info into base job class * Add versionadded directive to new property and class * Include new class in docs reference

fix: make unicode characters working well in load_table_from_json (#865)
ad9c802
Co-authored-by: Tim Swast <swast@google.com> Co-authored-by: Tres Seaver <tseaver@palladion.com>

gcf-owl-botBotand others added 6 commits October 8, 2021 18:34

chore(python): Add kokoro configs for python 3.10 samples testing (#1010
c8124e9
)

Merge remote-tracking branch 'upstream/main' into sync-v3
ed9b6cf

Merge remote-tracking branch 'upstream/sync-v3' into sync-v3
294990a

Merge remote-tracking branch 'upstream/main' into sync-v3
d448d0e

tswast added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Oct 12, 2021

tswastand others added 4 commits October 12, 2021 20:16

feat: allow queryJob.result() to be called on a dryRun (#1015)
685f06a
* feat: allow queryJob.result() to be called on a dryRun * update to return EmptyRowIterator

Merge remote-tracking branch 'upstream/main' into sync-v3
50753cc

tswast removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Oct 15, 2021

docs: document ScriptStatistics and other missing resource classes (#…
6679109
…1023) While implementing Sessions, I noticed several missing classes in the reference docs. Using "automodule" since experience has shown we often forget to add new classes to `docs/reference.rst`.

plamut suggested changes Oct 16, 2021
View reviewed changes

plamut added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Oct 16, 2021

gcf-owl-botBotand others added 2 commits October 25, 2021 18:04

Merge remote-tracking branch 'upstream/main' into sync-v3
c67377a

tswast requested a review from plamut October 26, 2021 15:26

plamut removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Oct 28, 2021

plamut approved these changes Oct 28, 2021
View reviewed changes

plamut merged commit 6706678 into v3Oct 28, 2021

plamut deleted the sync-v3 branch October 28, 2021 06:57

Navigation Menu

Uh oh!

chore: sync v3 with main - #1004

chore: sync v3 with main#1004
plamut merged 121 commits into
v3from
sync-v3

tswast commented Oct 6, 2021

Uh oh!

tswast commented Oct 7, 2021

Uh oh!

tseaver commented Oct 7, 2021

Uh oh!

tswast commented Oct 7, 2021

Uh oh!

tswast commented Oct 12, 2021

Uh oh!

tswast commented Oct 15, 2021

Uh oh!

plamut commented Oct 16, 2021

Uh oh!

plamut left a comment

Uh oh!

plamutOct 16, 2021

Uh oh!

plamutOct 16, 2021

Uh oh!

tswastOct 26, 2021

Uh oh!

tswastOct 27, 2021

Uh oh!

plamut commented Oct 16, 2021

Uh oh!

plamut commented Oct 28, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

		"proto-plus": ("https://proto-plus-python.readthedocs.io/en/latest/", None),
		"protobuf": ("https://googleapis.dev/python/protobuf/latest/", None),

		# Remove unneeded intersphinx links, the library does not use any proto-generated code.
		s.replace("docs/conf.py", r'\s+"(proto-plus\|protobuf)":.*$', "")

Uh oh!

Conversation

tswast commented Oct 6, 2021

Uh oh!

tswast commented Oct 7, 2021

Uh oh!

tseaver commented Oct 7, 2021

Uh oh!

tswast commented Oct 7, 2021

Uh oh!

tswast commented Oct 12, 2021

Uh oh!

tswast commented Oct 15, 2021

Uh oh!

plamut commented Oct 16, 2021

Uh oh!

plamut left a comment

Choose a reason for hiding this comment

Uh oh!

plamutOct 16, 2021

Choose a reason for hiding this comment

Uh oh!

plamutOct 16, 2021

Choose a reason for hiding this comment

Uh oh!

tswastOct 26, 2021

Choose a reason for hiding this comment

Uh oh!

tswastOct 27, 2021

Choose a reason for hiding this comment

Uh oh!

plamut commented Oct 16, 2021

Uh oh!

plamut commented Oct 28, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants