Skip to content

fix: handle unsupported types and empty results in describe - #2506

Merged
shuoweil merged 4 commits into
mainfrom
shuowei-fix-describe-unsupported
Mar 13, 2026
Merged

fix: handle unsupported types and empty results in describe#2506
shuoweil merged 4 commits into
mainfrom
shuowei-fix-describe-unsupported

Conversation

@shuoweil

@shuoweilshuoweil commented Mar 11, 2026

Copy link
Copy Markdown
Contributor

The .describe() method attempts to compute metrics like mean, max, and unique on all columns. For MM complex types (structs representing images/videos), running COUNT(DISTINCT column) or mathematical aggregates is physically prohibited by BigQuery and raises syntax/type validation errors.

We limit aggregations on OBJ_REF_DTYPE and JSON columns to only a basic .count(), skipping unhashable/unsupported summary metrics.

Fixes #<452681068> 🦕

@shuoweil
shuoweil requested review from a team as code ownersMarch 11, 2026 20:21
@shuoweil
shuoweil requested a review from tswastMarch 11, 2026 20:21
@product-auto-labelproduct-auto-labelBot added the size: s Pull request size is small. label Mar 11, 2026
@product-auto-labelproduct-auto-labelBot added the api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. label Mar 11, 2026
@shuoweil
shuoweil requested review from TrevorBergeron and chelsea-lin and removed request for tswastMarch 11, 2026 20:21
@shuoweilshuoweil assigned shuoweil and unassigned mpovoaMar 11, 2026

def test_describe_empty_dataframe_returns_empty_dataframe(session):
df = session.read_gbq("SELECT 1 AS int_col LIMIT 0")
df = df.drop(columns=["int_col"])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Can we do df = bpd.DataFrame() for an empty DataFrame?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion, the testcase has been updated.

@shuoweil
shuoweil merged commit 2326ad6 into mainMar 13, 2026
24 of 25 checks passed
@shuoweil
shuoweil deleted the shuowei-fix-describe-unsupported branch March 13, 2026 22:04
tswast added a commit that referenced this pull request Mar 16, 2026
PR created by the Librarian CLI to initialize a release. Merging this PR
will auto trigger a release.
Librarian Version: v0.8.3
Language Image:
us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/python-librarian-generator@sha256:160860d189ff1c2f7515638478823712fa5b243e27ccc33a2728669fa1e2ed0c
<details><summary>bigframes: v2.38.0</summary>
##
[v2.38.0](v2.37.0...v2.38.0)
(2026-03-16)
### Features
* Add `bigframes.bigquery.rand()` function (#2501)
([5c43efb](5c43efb7))
* add `df.bigquery` pandas accessor (#2513)
([91b6c24](91b6c245))
* Add str, dt accessors to pd.col Expression objects (#2488)
([ce5de57](ce5de570))
* add bigquery.ml.get_insights function (#2493)
([d29a609](d29a6095))
* use EUC for AI IF, CLASSIFY, and SCORE when connection is not provided
(#2507)
([fe94910](fe94910a))
### Bug Fixes
* handle unsupported types and empty results in describe (#2506)
([2326ad6](2326ad6a))
* no longer automatically use anywidget in the `%%bqsql` magics (#2504)
([43353e2](43353e2b))
### Documentation
* Fix typo in ExperimentOptions class docstring (#2498)
([077cb2e](077cb2eb))
* add notebooks to user guide page (#2505)
([5cf3788](5cf37888))
</details>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigqueryIssues related to the googleapis/python-bigquery-dataframes API.size: sPull request size is small.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@shuoweil@chelsea-lin@mpovoa