Skip to content

ARROW-10293: [Rust] [DataFusion] Fixed benchmarks - #8452

Closed
jorgecarleitao wants to merge 1 commit into
apache:masterfrom
jorgecarleitao:bench
Closed

ARROW-10293: [Rust] [DataFusion] Fixed benchmarks#8452
jorgecarleitao wants to merge 1 commit into
apache:masterfrom
jorgecarleitao:bench

Conversation

@jorgecarleitao

Copy link
Copy Markdown
Member

The benchmarks were only benchmarking planning, not execution, of the plans. This PR fixes this.

@jorgecarleitao

Copy link
Copy Markdown
MemberAuthor

FYI @andygrove , as 2.0.0' benchmarks not benchmarking anything atm :/

@github-actions

Copy link
Copy Markdown

@alambalamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR also changes some of the batch sizes -- which seems reasonable.


// display the relation
for _batch in results {}
rt.block_on(df.collect()).unwrap();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 this is the major change, right?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, the block is pick up the task. We also benchmark creating a runtime itself, but that was a very small compared to the task itself, so I left it there.

kszucs pushed a commit that referenced this pull request Oct 19, 2020
The benchmarks were only benchmarking planning, not execution, of the plans. This PR fixes this.
Closes#8452 from jorgecarleitao/bench
Authored-by: Jorge C. Leitao <jorgecarleitao@gmail.com>
Signed-off-by: Andy Grove <andygrove@nvidia.com>
@jorgecarleitao
jorgecarleitao deleted the bench branch October 28, 2020 04:16
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@jorgecarleitao@alamb@andygrove