Skip to content

feat: support INFORMATION_SCHEMA views in read_gbq - #1895

Merged
tswast merged 19 commits into
mainfrom
b374307132-information_schema
Oct 30, 2025
Merged

feat: support INFORMATION_SCHEMA views in read_gbq#1895
tswast merged 19 commits into
mainfrom
b374307132-information_schema

Conversation

@tswast

@tswasttswast commented Jul 9, 2025

Copy link
Copy Markdown
Collaborator

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes internal issue b/374307132 🦕

@product-auto-labelproduct-auto-labelBot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. labels Jul 9, 2025
tswast
tswast commented Jul 9, 2025
View reviewed changes
Comment threadbigframes/session/_io/bigquery/read_gbq_table.py Outdated
@tswast
tswast marked this pull request as ready for review July 9, 2025 23:05
@tswast
tswast requested review from a team and TrevorBergeronJuly 9, 2025 23:05
@tswasttswast changed the title feat: support INFORMATION_SCHEMA tables in read_gbqfeat: support INFORMATION_SCHEMA views in read_gbqJul 10, 2025
tswast
tswast commented Jul 10, 2025
View reviewed changes
Comment threadbigframes/session/_io/bigquery/read_gbq_table.py Outdated
@tswast

tswast commented Oct 23, 2025

Copy link
Copy Markdown
CollaboratorAuthor

429bea3 updates this PR to immediately run the query when getting an INFORMATION_SCHEMA table. I think this is now ready for review.

@product-auto-labelproduct-auto-labelBot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Oct 23, 2025
@product-auto-labelproduct-auto-labelBot added size: m Pull request size is medium. and removed size: l Pull request size is large. labels Oct 23, 2025
@tswast

This comment was marked as resolved.

Sign in to view
@jialuoo

This comment was marked as resolved.

Sign in to view
@tswast
tswast merged commit d97cafc into mainOct 30, 2025
23 of 25 checks passed
@tswast
tswast deleted the b374307132-information_schema branch October 30, 2025 14:36
@release-pleaserelease-pleaseBot mentioned this pull request Oct 30, 2025
Merged
shuoweil pushed a commit that referenced this pull request Oct 30, 2025
@tswast@shuoweil
feat: support INFORMATION_SCHEMA views inread_gbq(#1895)
a538c69
* feat: support INFORMATION_SCHEMA tables in read_gbq
* avoid storage semi executor
* use faster tables for peek tests
* more tests
* fix mypy
* Update bigframes/session/_io/bigquery/read_gbq_table.py
* immediately query for information_schema tables
* Fix mypy errors and temporarily update python version
* snapshot
* snapshot again
tswast added a commit that referenced this pull request Nov 4, 2025
@shuoweil@jialuoo@tswast
docs: update notebook for JSON subfields support in to_pandas_batches…
5663d2a
…() (#2138)
* change to ai.generate
* perf: Default to interactive display for SQL in anywidget mode
Previously, SQL queries in anywidget mode would fall back to deferred execution, showing a dry run instead of an interactive table.
This change modifies the display logic to directly use the anywidget interactive display for SQL queries, providing a more consistent and responsive user experience. A test case has been added to verify this behavior.
* fix: resolve double printing issue in anywidget mode
* feat: Add test case for STRUCT column in anywidget
Adds a test case to verify that a DataFrame with a STRUCT column is
correctly displayed in anywidget mode.
This test confirms that displaying a STRUCT column does not raise an
exception that would trigger the fallback to the deferred representation.
It mocks `IPython.display.display` to capture the `TableWidget` instance
and asserts that the rendered HTML contains the expected string
representation of the STRUCT data.
* fix presubmit
* Revert accidental changes to test_function.py
* revert accidental change to blob.py
* change return type
* add todo and revert change
* Revert "add todo and revert change"
This reverts commit 153e1d2.
* Add todo
* Fix: Handle JSON dtype in anywidget display
This commit fixes an AttributeError that occurred when displaying a
DataFrame with a JSON column in anywidget mode. The dtype check
was incorrect and has been updated. Additionally, the SQL compilation
for casting JSON to string has been corrected to use TO_JSON_STRING.
* revert a change
* revert a change
* Revert: Restore bigframes/dataframe.py to state from 42da847
* remove anywidget from early return, allow execution proceeds to _repr_html_()
* remove unnecessary changes
* remove redundant code change
* code style change
* tescase update
* revert a change
* final touch of notebook
* fix presumbit error
* remove invlaid test with anywidget bug fix
* fix presubmit
* fix polar complier
* Revert an unnecessary change
* apply the workaround to i/O layer
* Revert scalar_op_registry.py chnage
* remove unnecessary import
* Remove duplicate conversation
* revert changes to test_dataframe.py
* notebook update
* call API on local data for complier.py
* add more testcase
* modfiy polars import
* fix failed tests
* chore: Migrate minimum_op operator to SQLGlot (#2205)
* chore: Migrate round_op operator to SQLGlot (#2204)
This commit migrates the `round_op` operator from the Ibis compiler to the SQLGlot compiler.
* fix: Improve error handling in blob operations (#2194)
* add error handling for audio_transcribe
* add error handling for pdf functions
* add eror handling for image functions
* final touch
* restore rename
* update notebook to better reflect our new code change
* return None on error with verbose=False for image functions
* define typing module in udf
* only use local variable
* Refactor code
* refactor: update geo "spec" and split geo ops in ibis compiler (#2208)
* feat: support INFORMATION_SCHEMA views in `read_gbq` (#1895)
* feat: support INFORMATION_SCHEMA tables in read_gbq
* avoid storage semi executor
* use faster tables for peek tests
* more tests
* fix mypy
* Update bigframes/session/_io/bigquery/read_gbq_table.py
* immediately query for information_schema tables
* Fix mypy errors and temporarily update python version
* snapshot
* snapshot again
* Revert: Unwanted code changes
* Revert "Revert: Unwanted code changes"
This reverts commit db5d8ea.
* revert 1 files to match main branch
* Correctly display DataFrames with JSON columns in anywidget
* add mis-deleted comment back
* revert unnecessary change
* move helper function to dtypes.py
* revert unnecessary testcase change
* Improve JSON type handling for to_gbq and to_pandas_batches
* Remove unnecessary comment
* Revert bigframes/dtypes.py and mypy.ini to main branch version
---------
Co-authored-by: jialuoo <jialuo@google.com>
Co-authored-by: Tim Sweña (Swast) <swast@google.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@TrevorBergeronTrevorBergeronTrevorBergeron approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@jialuoojialuoo

Labels

api: bigqueryIssues related to the googleapis/python-bigquery-dataframes API.size: mPull request size is medium.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@tswast@jialuoo@TrevorBergeron