Skip to content

Minor: return internal error rather than panic on unexpected error in COUNT DISTINCT - #9712

Merged
alamb merged 4 commits into
apache:mainfrom
alamb:alamb/do_not_panic
Mar 21, 2024
Merged

Minor: return internal error rather than panic on unexpected error in COUNT DISTINCT#9712
alamb merged 4 commits into
apache:mainfrom
alamb:alamb/do_not_panic

Conversation

@alamb

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Follow on to #9679

Rationale for this change

@Omega359 pointed out that the use of expect would panic on unexpected results: #9679 (comment)

What changes are included in this PR?

Change a panic into an internal error

Are these changes tested?

No (it is "impossible" to hit this code)

Are there any user-facing changes?

Better UX on unexpected error

@alamb
alamb marked this pull request as ready for review March 20, 2024 17:30
@github-actionsgithub-actionsBot added the physical-expr Changes to the physical-expr crates label Mar 20, 2024
@DandandanDandandan changed the title Minor: return internal error rather than panic on unexpected error in COUNT DISTICTMinor: return internal error rather than panic on unexpected error in COUNT DISTINCTMar 20, 2024
Comment threaddatafusion/physical-expr/src/aggregate/count_distinct/mod.rs Outdated
@@ -268,8 +268,11 @@ impl Accumulator for DistinctCountAccumulator {
let array = &states[0];
let list_array = array.as_list::<i32>();
for inner_array in list_array.iter() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we just filter out Nones? just thinking

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think filtering out nulls would potentially silently ignore errors

I think the rationale goes like

  1. This code is used to combine multiple partial results (the results of calling Self::state())
  2. Since Self::state never returns None -- it always returns Ok(vec![ScalarValue::List(arr)]) then this merge code should never receive a None

Comment threaddatafusion/physical-expr/src/aggregate/count_distinct/mod.rs Outdated

@compheadcomphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm thanks @alamb for explanation

@alamb
alamb merged commit 6d74025 into apache:mainMar 21, 2024
@alamb

Copy link
Copy Markdown
ContributorAuthor

Thanks for the review @comphead

@alamb
alamb deleted the alamb/do_not_panic branch March 21, 2024 19:08
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-exprChanges to the physical-expr crates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@alamb@comphead