Skip to content

Add baseline metrics to SortPreservingMergeExec - #948

Merged
alamb merged 2 commits into
apache:masterfrom
alamb:alamb/annotate_merge_et_al
Aug 27, 2021
Merged

Add baseline metrics to SortPreservingMergeExec#948
alamb merged 2 commits into
apache:masterfrom
alamb:alamb/annotate_merge_et_al

Conversation

@alamb

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Next part #866

Builds on #938 so review that first

Rationale for this change

We want basic understanding of where a plan's time is spent and in what operators. See #866 for more details

What changes are included in this PR?

  1. Instrument SortPreservingMergeExec using the API from Add BaselineMetrics, Timestamp metrics, add for CoalescePartitionsExec, rename output_time -> elapsed_compute #909

Are there any user-facing changes?

More fields in EXPLAIN ANALYZE are now filled out

@alambalamb changed the title Add metrics to `SortPreservingMergeExecAdd metrics to SortPreservingMergeExecAug 25, 2021
@alambalamb changed the title Add metrics to SortPreservingMergeExecAdd baselie metrics to SortPreservingMergeExecAug 26, 2021
@alambalamb changed the title Add baselie metrics to SortPreservingMergeExecAdd baseline metrics to SortPreservingMergeExecAug 26, 2021
@alamb
alambforce-pushed the alamb/annotate_merge_et_al branch from 418072c to be9428aCompareAugust 26, 2021 12:38
@alamb
alambforce-pushed the alamb/annotate_merge_et_al branch from a730a2f to dcc601eCompareAugust 26, 2021 13:46
@alamb
alamb marked this pull request as ready for review August 26, 2021 17:25

@NGA-TRANNGA-TRAN left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding these metrics @alamb. I can't wait to play with it.

// Now, validate metrics
let metrics = merge.metrics().unwrap();

assert!(metrics.output_rows().unwrap() > 0);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can verify the exact output_rows here, right?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is a good point. Will do

@alamb
alamb merged commit 9a07f1d into apache:masterAug 27, 2021
@alamb
alamb deleted the alamb/annotate_merge_et_al branch August 27, 2021 19:34
@houqphouqp added the enhancement New feature or request label Aug 30, 2021
unkloud pushed a commit to unkloud/datafusion that referenced this pull request Mar 23, 2025
HairstonE pushed a commit to HairstonE/datafusion that referenced this pull request Oct 7, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementNew feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@alamb@houqp@NGA-TRAN