Skip to content

ARROW-11107: [Rust] Faster Length kernel (10%) - #9077

Closed
jorgecarleitao wants to merge 5 commits into
apache:masterfrom
jorgecarleitao:length_faster
Closed

ARROW-11107: [Rust] Faster Length kernel (10%)#9077
jorgecarleitao wants to merge 5 commits into
apache:masterfrom
jorgecarleitao:length_faster

Conversation

@jorgecarleitao

@jorgecarleitaojorgecarleitao commented Jan 2, 2021

Copy link
Copy Markdown
Member

This is a corollary of the API PRed in #9076 , and how we can use it to efficiently build Buffer without even using the MutableBuffer. The main idea here is that we can write

 let lengths: Buffer = slice
.windows(2)
.map(|offset| offset[1] - offset[0])
.collect();

and this (efficiently) builds a Buffer from the offset differences.

Bench:

length time: [3.2345 us 3.2457 us 3.2602 us] change: [-12.072% -11.361% -10.654%] (p = 0.00 < 0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
4 (4.00%) high mild
6 (6.00%) high severe

@github-actions

Copy link
Copy Markdown

@codecov-io

Copy link
Copy Markdown

Codecov Report

Merging #9077 (c1badea) into master (4b7cdcb) will decrease coverage by 0.05%.
The diff coverage is 93.50%.

Impacted file tree graph

@@ Coverage Diff @@## master #9077 +/- ##
==========================================
- Coverage 82.61% 82.56% -0.06% 
==========================================
Files 203 204 +1 Lines 50140 50091 -49 ==========================================
- Hits 41422 41356 -66 - Misses 8718 8735 +17 
Impacted FilesCoverage Δ
rust/arrow/src/bytes.rs53.12% <ø> (-5.21%)⬇️
rust/arrow/src/compute/kernels/aggregate.rs74.93% <ø> (-0.07%)⬇️
rust/arrow/src/datatypes.rs75.02% <ø> (ø)
rust/arrow/src/array/transform/list.rs86.20% <66.66%> (+2.33%)⬆️
rust/arrow/src/array/transform/mod.rs88.29% <83.33%> (ø)
rust/arrow/src/array/builder.rs85.23% <86.79%> (-0.67%)⬇️
rust/arrow/src/buffer.rs95.53% <88.75%> (-2.19%)⬇️
rust/arrow/src/array/array_binary.rs90.54% <100.00%> (ø)
rust/arrow/src/array/array_list.rs92.66% <100.00%> (-0.45%)⬇️
rust/arrow/src/array/array_primitive.rs92.28% <100.00%> (-0.04%)⬇️
... and 25 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4b7cdcb...c1badea. Read the comment docs.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@jorgecarleitao@codecov-io