Simplify HashMap Bucket interface by arthurprs · Pull Request #40561 · rust-lang/rust

arthurprs · 2017-03-15T22:37:10Z

Simplify HashMap Bucket interface
Store capacity_mask instead of capacity
Move bucket index into RawBucket
Valid bucket index is now always within [0..table_capacity)
Simplify iterators by moving logic into RawBuckets
Clone RawTable using RawBucket
Make retain aware of the number of elements

The idea was to put idx in RawBucket instead of the other Bucket types and simplify next() and prev() as much as possible. The rest was a side-effect of that change, except maybe the last 2.

This change makes iteration and other next/prev() heavy operations noticeably faster. Clone is way faster.

➜ hashmap2 git:(adapt) ✗ cargo benchcmp pre:: adp:: bench.txt name pre:: ns/iter adp:: ns/iter diff ns/iter diff % clone_10_000 74,364 39,736 -34,628 -46.57% grow_100_000 8,343,553 8,233,785 -109,768 -1.32% grow_10_000 817,825 723,958 -93,867 -11.48% grow_big_value_100_000 18,418,979 17,906,186 -512,793 -2.78% grow_big_value_10_000 1,219,242 1,103,334 -115,908 -9.51% insert_1000 74,546 58,343 -16,203 -21.74% insert_100_000 6,743,770 6,238,017 -505,753 -7.50% insert_10_000 798,079 719,123 -78,956 -9.89% insert_1_000_000 275,215,605 266,975,875 -8,239,730 -2.99% insert_int_bigvalue_10_000 1,517,387 1,419,838 -97,549 -6.43% insert_str_10_000 316,179 278,896 -37,283 -11.79% insert_string_10_000 770,927 747,449 -23,478 -3.05% iter_keys_100_000 386,099 333,104 -52,995 -13.73% iterate_100_000 387,320 355,707 -31,613 -8.16% lookup_100_000 206,757 193,063 -13,694 -6.62% lookup_100_000_unif 219,366 193,180 -26,186 -11.94% lookup_1_000_000 206,456 205,716 -740 -0.36% lookup_1_000_000_unif 659,934 629,659 -30,275 -4.59% lru_sim 20,194,334 18,442,149 -1,752,185 -8.68% merge_shuffle 1,168,044 1,063,055 -104,989 -8.99%

Note 2: I may have messed up porting the diff, let's see what CI says.

rust-highfive · 2017-03-15T22:37:21Z

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

gereeter · 2017-03-15T23:38:13Z

Why was this renamed? If it was to disambiguate with the method, I think hash_ would be more appropriate than _hash, since the standard practice is to use prefix _ to denote "unused."

It's for disambiguation. I think we can come up with a better name instead, moving the _ to the end hurts the eye. Like hash_start, hash_base, first_hash?

gereeter · 2017-03-15T23:51:24Z

This seems like a worse name, since it isn't clear which raw bucket is returned. Maybe first_raw_bucket or raw_bucket_first or stick with the original first_bucket_raw (presumably paired with bucket_at_raw).

I'll switch callers to raw_bucket_at(0).

gereeter · 2017-03-15T23:53:28Z

Could you add a comment noting that these functions are unsafe because you can safely create out-of-bounds buckets?

gereeter · 2017-03-15T23:57:09Z

If this is always within [0..table_capacity), could you add a comment mentioning that.

gereeter · 2017-03-15T23:58:05Z

You said in your description that bucket indices are in [0..table_capacity), but this is one past that. Did you mean [0..table_capacity]?

This is intentionally set to one past the end for the reverse style iterator. I will add a comment line on that.

gereeter · 2017-03-15T23:59:53Z

It seems odd to have an iterator named RawBuckets that doesn't return RawBuckets.

Good point, hopefully that won't affect performance negatively.

gereeter · 2017-03-16T00:00:29Z

Nit: could there be a blank line before this loop?

gereeter · 2017-03-16T00:00:49Z

Nit: could there be a blank line before this impl.

alexcrichton · 2017-03-16T05:46:45Z

@bors: delegate=gereeter

bors · 2017-03-16T05:46:46Z

✌️ @gereeter can now approve this pull request

arthurprs · 2017-03-16T09:41:33Z

Thanks for the review @gereeter. I'll updated the PR later.

bors · 2017-03-19T07:09:42Z

☔ The latest upstream changes (presumably #40538) made this pull request unmergeable. Please resolve the merge conflicts.

arthurprs · 2017-03-19T22:27:10Z

Rebased.

alexcrichton · 2017-03-20T14:28:31Z

cc @bluss, @pczarn

bors · 2017-03-25T02:08:01Z

☔ The latest upstream changes (presumably #40806) made this pull request unmergeable. Please resolve the merge conflicts.

arthurprs · 2017-04-04T12:28:20Z

(gentle ping)

pczarn · 2017-04-04T13:57:44Z

Is this invariant not as strict as the old one? Either way it's ok. You could add a comment that this assertion is simplified.

It's not. I'll updat the code to do the same check as before.

pczarn

Excellent. ~~Please make a rebase~~

arthurprs · 2017-04-04T17:38:17Z

Updated with @pczarn review.

* Store capacity_mask instead of capacity * Move bucket index into RawBucket * Bucket index is now always within [0..table_capacity) * Clone RawTable using RawBucket * Simplify iterators by moving logic into RawBuckets * Make retain aware of the number of elements

alexcrichton · 2017-04-05T00:17:53Z

@bors: r=pczarn

bors · 2017-04-05T00:17:54Z

📌 Commit f07ebd6 has been approved by pczarn

alexcrichton · 2017-04-05T00:17:58Z

@bors: delegate=pczarn

bors · 2017-04-05T00:17:59Z

✌️ @pczarn can now approve this pull request

bors · 2017-04-05T08:06:55Z

⌛ Testing commit f07ebd6 with merge b181396...

bors · 2017-04-05T08:08:25Z

💔 Test failed - status-appveyor

alexcrichton · 2017-04-05T16:26:23Z

@bors: retry * network failure #40474

…

On Wed, Apr 5, 2017 at 2:08 AM, bors ***@***.***> wrote: 💔 Test failed - status-appveyor <https://ci.appveyor.com/project/rust-lang/rust/build/1.0.2769> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#40561 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAD95G7JvqimISUpYtdiDu2ak4yAt2-Dks5rs0v6gaJpZM4Menm5> .

Simplify HashMap Bucket interface > Simplify HashMap Bucket interface > > * Store capacity_mask instead of capacity > * Move bucket index into RawBucket > * Valid bucket index is now always within [0..table_capacity) > * Simplify iterators by moving logic into RawBuckets > * Clone RawTable using RawBucket > * Make retain aware of the number of elements The idea was to put idx in RawBucket instead of the other Bucket types and simplify next() and prev() as much as possible. The rest was a side-effect of that change, except maybe the last 2. This change makes iteration and other next/prev() heavy operations noticeably faster. Clone is way faster. ``` ➜ hashmap2 git:(adapt) ✗ cargo benchcmp pre:: adp:: bench.txt name pre:: ns/iter adp:: ns/iter diff ns/iter diff % clone_10_000 74,364 39,736 -34,628 -46.57% grow_100_000 8,343,553 8,233,785 -109,768 -1.32% grow_10_000 817,825 723,958 -93,867 -11.48% grow_big_value_100_000 18,418,979 17,906,186 -512,793 -2.78% grow_big_value_10_000 1,219,242 1,103,334 -115,908 -9.51% insert_1000 74,546 58,343 -16,203 -21.74% insert_100_000 6,743,770 6,238,017 -505,753 -7.50% insert_10_000 798,079 719,123 -78,956 -9.89% insert_1_000_000 275,215,605 266,975,875 -8,239,730 -2.99% insert_int_bigvalue_10_000 1,517,387 1,419,838 -97,549 -6.43% insert_str_10_000 316,179 278,896 -37,283 -11.79% insert_string_10_000 770,927 747,449 -23,478 -3.05% iter_keys_100_000 386,099 333,104 -52,995 -13.73% iterate_100_000 387,320 355,707 -31,613 -8.16% lookup_100_000 206,757 193,063 -13,694 -6.62% lookup_100_000_unif 219,366 193,180 -26,186 -11.94% lookup_1_000_000 206,456 205,716 -740 -0.36% lookup_1_000_000_unif 659,934 629,659 -30,275 -4.59% lru_sim 20,194,334 18,442,149 -1,752,185 -8.68% merge_shuffle 1,168,044 1,063,055 -104,989 -8.99% ``` Note 2: I may have messed up porting the diff, let's see what CI says.

bors · 2017-04-05T23:01:45Z

⌛ Testing commit f07ebd6 with merge 1e10204...

arielb1 · 2017-04-05T23:02:36Z

@bors retry - prioritizing rollup

Rollup of 12 pull requests - Successful merges: #40479, #40561, #40709, #40815, #40909, #40927, #40943, #41015, #41028, #41052, #41054, #41065 - Failed merges:

rust-highfive assigned alexcrichton Mar 15, 2017

gereeter reviewed Mar 16, 2017
View reviewed changes

arthurprs force-pushed the hm-adapt2 branch from bc0ce25 to 97340c9Compare March 16, 2017 14:11

arthurprs force-pushed the hm-adapt2 branch from 97340c9 to 970bf5aCompare March 19, 2017 22:26

arthurprs force-pushed the hm-adapt2 branch from 970bf5a to 3d8b25dCompare March 20, 2017 12:35

arthurprs force-pushed the hm-adapt2 branch 2 times, most recently from e190879 to 39802b2Compare March 25, 2017 22:13

pczarn reviewed Apr 4, 2017
View reviewed changes

pczarn approved these changes Apr 4, 2017
View reviewed changes

arthurprs force-pushed the hm-adapt2 branch 2 times, most recently from 3451af6 to fdb88feCompare April 4, 2017 17:37

arthurprs force-pushed the hm-adapt2 branch from fdb88fe to f07ebd6Compare April 4, 2017 20:43

frewsxcv mentioned this pull request Apr 5, 2017
Rollup of 9 pull requests #41093
Closed

arielb1 mentioned this pull request Apr 5, 2017
Rollup of 12 pull requests #41094
Closed

arielb1 mentioned this pull request Apr 5, 2017
Rollup of 12 pull requests #41098
Merged

bors added a commit that referenced this pull request Apr 6, 2017
Auto merge of#41098- arielb1:rollup, r=arielb1
6cd15a0
Rollup of 12 pull requests - Successful merges: #40479, #40561, #40709, #40815, #40909, #40927, #40943, #41015, #41028, #41052, #41054, #41065 - Failed merges:

bors merged commit f07ebd6 into rust-lang:masterApr 6, 2017

arthurprs mentioned this pull request Apr 19, 2017
Decrease HashMap Load Factor #38003
Closed

arthurprs mentioned this pull request Aug 30, 2017
*: use FnvHashMap again tikv/tikv#2211
Merged

Uh oh!

Uh oh!

Conversation

arthurprs commented Mar 15, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rust-highfive commented Mar 15, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arthurprsMar 16, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alexcrichton commented Mar 16, 2017

Uh oh!

bors commented Mar 16, 2017

Uh oh!

arthurprs commented Mar 16, 2017

Uh oh!

bors commented Mar 19, 2017

Uh oh!

arthurprs commented Mar 19, 2017

Uh oh!

alexcrichton commented Mar 20, 2017

Uh oh!

bors commented Mar 25, 2017

Uh oh!

arthurprs commented Apr 4, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arthurprsApr 4, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pczarn left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arthurprs commented Apr 4, 2017

Uh oh!

alexcrichton commented Apr 5, 2017

Uh oh!

bors commented Apr 5, 2017

Uh oh!

alexcrichton commented Apr 5, 2017

Uh oh!

bors commented Apr 5, 2017

Uh oh!

bors commented Apr 5, 2017

Uh oh!

bors commented Apr 5, 2017

Uh oh!

alexcrichton commented Apr 5, 2017 via email

Uh oh!

bors commented Apr 5, 2017

Uh oh!

arielb1 commented Apr 5, 2017

arthurprs commented Mar 15, 2017•
edited
Loading

arthurprsMar 16, 2017•
edited
Loading

arthurprs commented Apr 4, 2017•
edited
Loading

arthurprsApr 4, 2017•
edited
Loading

pczarn left a comment •
edited
Loading