Uh oh!
There was an error while loading. Please reload this page.
feat: delegate ReadRowsPage.to_arrow to pandas_gbq.arrow - #17938
Conversation
There was a problem hiding this comment.
Code Review
This pull request deprecates direct retrieval of Arrow record batches via google-cloud-bigquery-storage and delegates this functionality to a new arrow submodule in pandas-gbq. The review feedback highlights several critical issues: 1) pyarrow.RecordBatch does not have a from_batches method, so from_arrays should be used instead; 2) importing pyarrow at the top level of pandas_gbq/arrow.py will break imports for users who do not have this optional dependency installed, requiring a conditional import and string type annotations; and 3) reader.read_next_batch() can raise a StopIteration exception if the stream is empty, which needs to be handled gracefully.
Uh oh!
There was an error while loading. Please reload this page.
| """Arrow integration submodule for pandas-gbq.""" | ||
| from typing import Any, Optional | ||
| import pyarrow as pa |
There was a problem hiding this comment.
Since pyarrow is an optional dependency for pandas-gbq, importing it at the top level of this module will cause import pandas_gbq to fail with an ImportError for any user who does not have pyarrow installed. We should wrap the import in a try-except block and raise a descriptive error only when the function is called. Additionally, when assigning None in the except ImportError block, type: ignore[assignment] is necessary to prevent a mypy error.
try:
importpyarrowaspaexceptImportError:
pa=None# type: ignore[assignment]References
- When handling an optional dependency by assigning
Nonein anexcept ImportErrorblock,type: ignore[assignment]is necessary on the assignment to prevent a mypy error, as usingtyping.TYPE_CHECKINGalone is insufficient.
There was a problem hiding this comment.
Fixed. Wrapped the optional pyarrow import in a try...except ImportError block with pa = None # type: ignore[assignment] in pandas_gbq/arrow.py, and added a runtime check in from_read_rows_response that raises a descriptive ImportError when pyarrow is missing. Added test coverage in test_arrow.py.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
4e2ddd1 to
a63af8fCompareUh oh!
There was an error while loading. Please reload this page.
🤖 I have created a release *beep* *boop* --- <details><summary>bigquery-magics: 0.15.1</summary> ## [0.15.1](bigquery-magics-v0.15.0...bigquery-magics-v0.15.1) (2026-08-06) ### Bug Fixes * bump grpcio to 1.59.0; require Python 3.10+ ([#17351](#17351)) ([a53487a](a53487a)) * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>gapic-generator: 1.38.0</summary> ## [1.38.0](gapic-generator-v1.37.1...gapic-generator-v1.38.0) (2026-08-06) ### Features * **generator:** delegate REST transcoding to google-api-core ([#17766](#17766)) ([7b05aab](7b05aab)) * **generator:** gapic generator centralization routing ([#17816](#17816)) ([32a2442](32a2442)) ### Bug Fixes * add typing for header_params ([#17914](#17914)) ([9e98b93](9e98b93)) * **api-core:** use truthiness check in setup_request_id to support proto-plus messages ([#18000](#18000)) ([ad8f93c](ad8f93c)) * avoid retaining routing parameter instances in cache ([#17961](#17961)) ([f64ada2](f64ada2)) * bump aiohttp from 3.13.5 to 3.14.3 in /packages/gapic-generator ([#17990](#17990)) ([6ff5815](6ff5815)) * bump cryptography from 48.0.1 to 50.0.0 in /packages/gapic-generator ([#17991](#17991)) ([d607f25](d607f25)) * bump grpcio to 1.59.0; require Python 3.10+ ([#17351](#17351)) ([a53487a](a53487a)) * **generator:** use flat_ref_types in test templates and delete remove-unused-imports ([#17900](#17900)) ([395f764](395f764)) * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) * resolve collision with reserved words in samples ([#17912](#17912)) ([588cda9](588cda9)) * upgrade Protobuf and gRPC in WORKSPACE ([#17882](#17882)) ([5b5ece5](5b5ece5)) </details> <details><summary>gcp-sphinx-docfx-yaml: 3.3.1</summary> ## [3.3.1](gcp-sphinx-docfx-yaml-v3.3.0...gcp-sphinx-docfx-yaml-v3.3.1) (2026-08-06) ### Bug Fixes * bump grpcio to 1.59.0; require Python 3.10+ ([#17351](#17351)) ([a53487a](a53487a)) * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-ads-admanager: 0.10.1</summary> ## [0.10.1](google-ads-admanager-v0.10.0...google-ads-admanager-v0.10.1) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-api-core: 2.34.0</summary> ## [2.34.0](google-api-core-v2.33.0...google-api-core-v2.34.0) (2026-08-06) ### Features * Add Feature Gating configuration helpers. ([#17524](#17524)) ([eceea95](eceea95)) * **api-core:** centralize rest transcoding helpers ([#17765](#17765)) ([4f21b8b](4f21b8b)) ### Bug Fixes * **api-core:** use truthiness check in setup_request_id to support proto-plus messages ([#18000](#18000)) ([ad8f93c](ad8f93c)) * bump grpcio to 1.59.0; require Python 3.10+ ([#17351](#17351)) ([a53487a](a53487a)) * deduplicate x-goog-api-client headers ([#17616](#17616)) ([6167e41](6167e41)) * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-apps-chat: 0.10.4</summary> ## [0.10.4](google-apps-chat-v0.10.3...google-apps-chat-v0.10.4) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-auth: 2.56.3</summary> ## [2.56.3](google-auth-v2.56.2...google-auth-v2.56.3) (2026-08-06) ### Bug Fixes * **auth:** avoid creating mTLS SSL context for custom async transports ([#17825](#17825)) ([fbe33f9](fbe33f9)), refs [#17622](#17622) * **auth:** only trigger mTLS certificate rotation on mTLS endpoints ([#17928](#17928)) ([f7b49ea](f7b49ea)) * **auth:** properly extract stdout from gnubby webauthn plugin failures ([#17885](#17885)) ([744e826](744e826)) * deduplicate x-goog-api-client headers ([#17616](#17616)) ([6167e41](6167e41)) * **oauth2:** avoid redundant JWKS network fetches ([#17891](#17891)) ([de53298](de53298)) ### Performance Improvements * **auth:** use generator expression in any() to allow short-circuiting ([735e565](735e565)) * **auth:** use generator expression in any() to allow short-circuiting ([#17937](#17937)) ([735e565](735e565)) </details> <details><summary>google-auth-httplib2: 0.4.1</summary> ## [0.4.1](google-auth-httplib2-v0.4.0...google-auth-httplib2-v0.4.1) (2026-08-06) ### Bug Fixes * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-cloud-access-context-manager: 0.6.1</summary> ## [0.6.1](google-cloud-access-context-manager-v0.6.0...google-cloud-access-context-manager-v0.6.1) (2026-08-06) ### Bug Fixes * bump grpcio to 1.59.0; require Python 3.10+ ([#17351](#17351)) ([a53487a](a53487a)) * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-cloud-agentidentitycredentials: 0.1.1</summary> ## [0.1.1](google-cloud-agentidentitycredentials-v0.1.0...google-cloud-agentidentitycredentials-v0.1.1) (2026-08-06) ### Features * **google/cloud/agentidentitycredentials/v1beta:** add google-cloud-agentidentitycredentials v1beta ([#17898](#17898)) ([b692dae](b692dae)) * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-cloud-audit-log: 0.6.1</summary> ## [0.6.1](google-cloud-audit-log-v0.6.0...google-cloud-audit-log-v0.6.1) (2026-08-06) ### Bug Fixes * bump grpcio to 1.59.0; require Python 3.10+ ([#17351](#17351)) ([a53487a](a53487a)) * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-cloud-backupdr: 0.10.1</summary> ## [0.10.1](google-cloud-backupdr-v0.10.0...google-cloud-backupdr-v0.10.1) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-cloud-batch: 0.22.2</summary> ## [0.22.2](google-cloud-batch-v0.22.1...google-cloud-batch-v0.22.2) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-cloud-biglake: 0.5.1</summary> ## [0.5.1](google-cloud-biglake-v0.5.0...google-cloud-biglake-v0.5.1) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-cloud-biglake-hive: 0.3.2</summary> ## [0.3.2](google-cloud-biglake-hive-v0.3.1...google-cloud-biglake-hive-v0.3.2) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-cloud-bigquery: 3.43.0</summary> ## [3.43.0](google-cloud-bigquery-v3.42.3...google-cloud-bigquery-v3.43.0) (2026-08-06) ### Features * add pandas-gbq capability helper ([#17957](#17957)) ([2207ca6](2207ca6)) ### Bug Fixes * **bigquery:** Fix bigquery socket leak ([#17953](#17953)) ([8c26b38](8c26b38)) * remove stray debug print in RangeQueryParameter constructor ([#17973](#17973)) ([fe7bfd0](fe7bfd0)) ### Documentation * add connector libraries overview table to package README ([#17939](#17939)) ([71bc622](71bc622)) * correct return type in CellDataParser.time_to_py docstring ([#17972](#17972)) ([bd1e224](bd1e224)) </details> <details><summary>google-cloud-bigquery-storage: 2.40.0</summary> ## [2.40.0](google-cloud-bigquery-storage-v2.39.0...google-cloud-bigquery-storage-v2.40.0) (2026-08-06) ### Features * delegate ReadRowsPage.to_arrow to pandas_gbq.arrow ([#17938](#17938)) ([aedc66f](aedc66f)) ### Bug Fixes * bump grpcio to 1.59.0; require Python 3.10+ ([#17351](#17351)) ([a53487a](a53487a)) </details> <details><summary>google-cloud-binary-authorization: 1.19.1</summary> ## [1.19.1](google-cloud-binary-authorization-v1.19.0...google-cloud-binary-authorization-v1.19.1) (2026-08-06) ### Bug Fixes * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-cloud-build: 3.38.1</summary> ## [3.38.1](google-cloud-build-v3.38.0...google-cloud-build-v3.38.1) (2026-08-06) ### Bug Fixes * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-cloud-commerceproducer: 0.1.1</summary> ## [0.1.1](google-cloud-commerceproducer-v0.1.0...google-cloud-commerceproducer-v0.1.1) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-cloud-containeranalysis: 2.22.1</summary> ## [2.22.1](google-cloud-containeranalysis-v2.22.0...google-cloud-containeranalysis-v2.22.1) (2026-08-06) ### Bug Fixes * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-cloud-core: 2.6.1</summary> ## [2.6.1](google-cloud-core-v2.6.0...google-cloud-core-v2.6.1) (2026-08-06) ### Bug Fixes * bump grpcio to 1.59.0; require Python 3.10+ ([#17351](#17351)) ([a53487a](a53487a)) * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-cloud-documentai-toolbox: 0.17.2</summary> ## [0.17.2](google-cloud-documentai-toolbox-v0.17.1...google-cloud-documentai-toolbox-v0.17.2) (2026-08-06) ### Bug Fixes * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-cloud-filestore: 1.17.1</summary> ## [1.17.1](google-cloud-filestore-v1.17.0...google-cloud-filestore-v1.17.1) (2026-08-06) ### Bug Fixes * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-cloud-firestore: 2.28.1</summary> ## [2.28.1](google-cloud-firestore-v2.28.0...google-cloud-firestore-v2.28.1) (2026-08-06) ### Bug Fixes * **firestore:** BulkWriter pop from an empty deque ([#17490](#17490)) ([8e826f0](8e826f0)) * **firestore:** preserve async limit_to_last ordering ([#17879](#17879)) ([caf2fdb](caf2fdb)) </details> <details><summary>google-cloud-gke-hub: 1.25.1</summary> ## [1.25.1](google-cloud-gke-hub-v1.25.0...google-cloud-gke-hub-v1.25.1) (2026-08-06) ### Bug Fixes * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-cloud-iam: 2.24.1</summary> ## [2.24.1](google-cloud-iam-v2.24.0...google-cloud-iam-v2.24.1) (2026-08-06) ### Bug Fixes * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-cloud-logging: 3.16.2</summary> ## [3.16.2](google-cloud-logging-v3.16.1...google-cloud-logging-v3.16.2) (2026-08-06) ### Bug Fixes * bump grpcio to 1.59.0; require Python 3.10+ ([#17351](#17351)) ([a53487a](a53487a)) * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-cloud-memorystore: 0.5.4</summary> ## [0.5.4](google-cloud-memorystore-v0.5.3...google-cloud-memorystore-v0.5.4) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-cloud-ndb: 2.5.1</summary> ## [2.5.1](google-cloud-ndb-v2.5.0...google-cloud-ndb-v2.5.1) (2026-08-06) ### Bug Fixes * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-cloud-network-connectivity: 2.17.0</summary> ## [2.17.0](google-cloud-network-connectivity-v2.16.0...google-cloud-network-connectivity-v2.17.0) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-cloud-network-services: 0.10.2</summary> ## [0.10.2](google-cloud-network-services-v0.10.1...google-cloud-network-services-v0.10.2) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) * update googleapis and regenerate ([#17933](#17933)) ([f7a23a0](f7a23a0)) </details> <details><summary>google-cloud-pubsub: 2.39.1</summary> ## [2.39.1](google-cloud-pubsub-v2.39.0...google-cloud-pubsub-v2.39.1) (2026-08-06) ### Bug Fixes * bump grpcio to 1.59.0; require Python 3.10+ ([#17351](#17351)) ([a53487a](a53487a)) </details> <details><summary>google-cloud-quotas: 0.6.2</summary> ## [0.6.2](google-cloud-quotas-v0.6.1...google-cloud-quotas-v0.6.2) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-cloud-redis-cluster: 0.5.2</summary> ## [0.5.2](google-cloud-redis-cluster-v0.5.1...google-cloud-redis-cluster-v0.5.2) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-cloud-service-control: 1.21.0</summary> ## [1.21.0](google-cloud-service-control-v1.20.0...google-cloud-service-control-v1.21.0) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-cloud-spanner: 3.69.1</summary> ## [3.69.1](google-cloud-spanner-v3.69.0...google-cloud-spanner-v3.69.1) (2026-08-06) ### Bug Fixes * bump grpcio to 1.59.0; require Python 3.10+ ([#17351](#17351)) ([a53487a](a53487a)) * **metrics:** fix GFE and AFE metrics publishing ([#17561](#17561)) ([abf1178](abf1178)) * **spanner:** escape embedded backticks in dbapi escape_name ([#17810](#17810)) ([c8b0b28](c8b0b28)) * **spanner:** implement dict protocol and nested unwrapping for JsonObject ([#17915](#17915)) ([06c1f05](06c1f05)), refs [#15870](#15870) </details> <details><summary>google-cloud-storage: 3.13.1</summary> ## [3.13.1](google-cloud-storage-v3.13.0...google-cloud-storage-v3.13.1) (2026-08-06) ### Bug Fixes * bump grpcio to 1.59.0; require Python 3.10+ ([#17351](#17351)) ([a53487a](a53487a)) * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-cloud-storage-control: 1.13.0</summary> ## [1.13.0](google-cloud-storage-control-v1.12.0...google-cloud-storage-control-v1.13.0) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-cloud-support: 0.5.2</summary> ## [0.5.2](google-cloud-support-v0.5.1...google-cloud-support-v0.5.2) (2026-08-06) ### Features * update googleapis and regenerate ([#17933](#17933)) ([f7a23a0](f7a23a0)) </details> <details><summary>google-cloud-tasks: 2.24.0</summary> ## [2.24.0](google-cloud-tasks-v2.23.0...google-cloud-tasks-v2.24.0) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-cloud-testutils: 1.9.2</summary> ## [1.9.2](google-cloud-testutils-v1.9.1...google-cloud-testutils-v1.9.2) (2026-08-06) ### Bug Fixes * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>google-developer-knowledge: 0.1.1</summary> ## [0.1.1](google-developer-knowledge-v0.1.0...google-developer-knowledge-v0.1.1) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-maps-navconnect: 0.2.1</summary> ## [0.2.1](google-maps-navconnect-v0.2.0...google-maps-navconnect-v0.2.1) (2026-08-06) ### Features * update googleapis and regenerate ([#17893](#17893)) ([e70ab6f](e70ab6f)) </details> <details><summary>google-resumable-media: 2.10.1</summary> ## [2.10.1](google-resumable-media-v2.10.0...google-resumable-media-v2.10.1) (2026-08-06) ### Bug Fixes * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>googleapis-common-protos: 1.75.1</summary> ## [1.75.1](googleapis-common-protos-v1.75.0...googleapis-common-protos-v1.75.1) (2026-08-06) ### Bug Fixes * bump grpcio to 1.59.0; require Python 3.10+ ([#17351](#17351)) ([a53487a](a53487a)) * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>grafeas: 1.24.0</summary> ## [1.24.0](grafeas-v1.23.0...grafeas-v1.24.0) (2026-08-06) ### Features * update googleapis and regenerate ([#17933](#17933)) ([f7a23a0](f7a23a0)) </details> <details><summary>grpc-google-iam-v1: 0.14.5</summary> ## [0.14.5](grpc-google-iam-v1-v0.14.4...grpc-google-iam-v1-v0.14.5) (2026-08-06) ### Bug Fixes * bump grpcio to 1.59.0; require Python 3.10+ ([#17351](#17351)) ([a53487a](a53487a)) * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>proto-plus: 1.28.3</summary> ## [1.28.3](proto-plus-v1.28.2...proto-plus-v1.28.3) (2026-08-06) ### Bug Fixes * bump grpcio to 1.59.0; require Python 3.10+ ([#17351](#17351)) ([a53487a](a53487a)) * **proto-plus:** add context to TypeErrors during message manipulation ([#17682](#17682)) ([08f21a6](08f21a6)) * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> <details><summary>sqlalchemy-bigquery: 1.17.2</summary> ## [1.17.2](sqlalchemy-bigquery-v1.17.1...sqlalchemy-bigquery-v1.17.2) (2026-08-06) ### Bug Fixes * require Protobuf 6.33.5+ ([#17743](#17743)) ([d267342](d267342)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --------- Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
) Following the addition of `PendingDeprecationWarning` to `ReadRowsPage.to_arrow()` in #17938, this PR adds `PendingDeprecationWarning` to `ReadRowsPage.to_dataframe()` in `google-cloud-bigquery-storage`. This notifies developers to adopt direct `pandas-gbq` APIs (`pandas_gbq.read_gbq()` or `pandas_gbq.pandas.read_bigquery_stream_batches()`) ahead of future deprecation phases. Fixes #< 526614511 > 🦕
Emits a deprecation warning on
ReadRowsPage.to_arrow()and delegates Arrow RecordBatch decoding topandas_gbq.arrow.from_read_rows_responsewhenpandas-gbqis installed, falling back to internal parsing otherwise.Changes Made
google/cloud/bigquery_storage_v1/reader.py: EmitsPendingDeprecationWarningand callspandas_gbq.arrow.from_read_rows_responsewith stream schema. Falls back to_stream_parser.to_arrowifpandas-gbqis uninstalled.tests/unit/test_reader_v1_arrow.py: Adds unit tests verifying delegation whenpandas-gbqis present and fallback behavior when uninstalled.Fixes #<540939659> 🦕