Skip to content

Implement streaming for specialized group by string aggregator - #9188

Closed
alamb wants to merge 2 commits into
apache:mainfrom
alamb:alamb/specialized_group_keys_with_streaming
Closed

Implement streaming for specialized group by string aggregator#9188
alamb wants to merge 2 commits into
apache:mainfrom
alamb:alamb/specialized_group_keys_with_streaming

Conversation

@alamb

@alambalamb commented Feb 10, 2024

Copy link
Copy Markdown
Contributor

This is a snapshot of the code for #8827

Related to #9195

This PR includes support for emit_to which I am confident is correct but is quite complicated and I didn't want to include in the first version

@github-actionsgithub-actionsBot added physical-expr Changes to the physical-expr crates core Core DataFusion crate labels Feb 10, 2024
@alamb
alambforce-pushed the alamb/specialized_group_keys_with_streaming branch from da68c45 to 489c4c1CompareFebruary 10, 2024 16:42
@alambalamb changed the title Implement stremaing for specialized group by stringsImplement streaming for specialized group by string aggregatorFeb 10, 2024
}
}

/// Removes the first n distinct values inserted into this set, in the order

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the code that removes only the first_n from the set and updates the state. It is pretty complicated and I am not sure the performance benefit is worth it

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does have significant test coverage, however

@alamb

alamb commented Mar 2, 2024

Copy link
Copy Markdown
ContributorAuthor

This thing is crazy complicated -- I don't plan to pursue it unless there is a real need to improve this performance

@alambalamb closed this Mar 2, 2024
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

coreCore DataFusion cratephysical-exprChanges to the physical-expr crates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@alamb