Compress most of spans to 32 bits

[petrochenkov]

As described in https://internals.rust-lang.org/t/rfc-compiler-refactoring-spans/1357/28

Closes#15594
r? @michaelwoerister

[petrochenkov]

TODO: Measure difference in actual memory consumption, especially on large crates.
82.68% of spans are "inlined" in the compiler and standard library crates, but that's an indirect measure and it needs to be verified on something else.

TODO2: Remove accidental RLS update

[petrochenkov]

Span is not Send/Sync if it uses thread-local interner and profile queries are sent to other threads, so I had to temporary remove spans from them.
I'll restore this back later, queries will have to use SpanData instead of Span (I hoped to keep it private, but it looks like there's no better choice :( ).

[petrochenkov]

If encoding/decoding with 2-bit tag looks too error-prone/unmaintainable (or turns out to be too slow), then I can redo this with 1-bit tag instead, it will reduce the persentage of inlined spans from 82.68% to 80.01% (on rustc/libstd data).

[petrochenkov]

@michaelwoerister
I'm not sure if Span can derive Hash or not (for incremental, etc).
Is hashing the 32-bit "index" enough for interned spans, or actual data must be hashed?

[arielb1]

Incremental uses HashStable anyway.

[arielb1]

i.e. this impl:

rust/src/librustc/ich/hcx.rs

Line 234 in ef227f5

impl<'a,'gcx,'tcx>HashStable<StableHashingContext<'a,'gcx,'tcx>>forSpan{

[michaelwoerister]

Yes, as @arielb1 says. We keep hashing for incr. comp. separate because often it's a lot expensive than what we need for a regular hash table and it often requires additional contextual information. So you don't have to worry about it.

You might want to provide a specialized implementation that hashes just one 32 bit value instead for bytes though (unless we do that anyway). But probably not worth the trouble.

[arielb1]

Why are you using a [u8; 4]? Is that to make it dealigned and allow slipping through smaller holes? Can't you use a #[repr(packed)] u32 and gain safety?

[petrochenkov]

@arielb1

Why are you using a [u8; 4]? Is that to make it dealigned and allow slipping through smaller holes?

Yes.

Can't you use a #[repr(packed)] u32 and gain safety?

Are reads from packed structs guaranteed to perform unaligned loads correctly?
Does #27060 happens only if reference is taken, passed somewhere and then dereferenced?
I wasn't sure, so I avoided packed. If packed works, I'd certainly prefer it to the union trick.

[Mark-Simulacrum]

We can run this through perf.rlo's benchmarks when ready, just run a try build and ping me on completion.

[arielb1]

Are reads from packed structs guaranteed to perform unaligned loads correctly?
Does #27060 happens only if reference is taken, passed somewhere and then dereferenced?

That's right - otherwise packed structs would be quite useless.

And #27060 (which I'm working on fixing right now) only means that some code that should be unsafe is allowed, so if you break it in any way that should be easy to fix.

[bors]

☔ The latest upstream changes (presumably #44654) made this pull request unmergeable. Please resolve the merge conflicts.

[michaelwoerister]

This looks awesome, @petrochenkov!
Like @arielb1, I'd also be interested in the performance impact.

[petrochenkov]

Updated.
@bors try (for performance testing)

[bors]

⌛ Trying commit f069c88 with merge 1125280...

[bors]

☀️ Test successful - status-travis
State: approved= try=True

[michaelwoerister]

@arielb1, I tried typing the merge commit hashes into perf.rlo compare but that didn't seem to work for me. Did you have a different method for performance comparison in mind?

[petrochenkov]

We can run this through perf.rlo's benchmarks when ready, just run a try build and ping me on completion.

ping @Mark-Simulacrum

[Mark-Simulacrum]

Results: http://perf.rust-lang.org/compare.html?commit_a=0701b37d97d08da7074ece7a7dcb4449498f4bfa&commit_b=11252805f7b1f43e0ae673ab1ed1757d801222ef&stat=instructions%3Au.

[llogiq]

I notice that the interner uses a default HashMap. Perhaps we can further speed it up using an optimized hash, given that the SpanData to hash is really small?

[petrochenkov]

I looked through the performance data a bit.

What we try to optimize is memory, i.e. max-rss.
The situation looks pretty good, tests use less memory, mostly.
Three regressions are regex-0.1.80@020-incr-from-scr..., regex-0.1.80@080-SparseSet, syntex-0.42.2@000-base. I'm not sure how to interpret this, on one hand regex and syntex are large crates so they can hit the limit for lo and start actively use interner, on the other hand many other tests using regex and syntex doesn't show regressions.

On the other side there are more regressions in speed. I don't know what exactly the tests measure (e.g. how cpu-clock and cycles-u are different), but the speed problem can be mitigated by 1) using 1-bit tag which makes encoding/decoding much more simple, 2) using faster hash function in interner as @llogiq suggested, 3) use span.data() + span_data.lo + span_data.hi more actively when you need e.g. span.lo() and span.hi() in the same place.

[michaelwoerister]

Many of the tests showing regressions are ones with incremental compilation activated. Incremental compilation has to expand all spans to file:line:col during change detection, so it will do more decoding than other cases. That might explain why these tests show up. That being said, in the future we will probably change how exactly spans are treated by change detection, so I don't think we should give too much weight to the incr. comp. tests right now.

Great find, @llogiq! This should definitely use rustc_data_structures::fx::FxHashMap.

[petrochenkov]

@bors try
Let's try 1-bit tag before doing other optimizations.

[bors]

🔒 Merge conflict

[petrochenkov]

@bors try

[bors]

⌛ Trying commit cb1158f with merge 39d2aa2...

[Mark-Simulacrum]

http://perf.rust-lang.org/compare.html?commit_a=17f56c549c35bb2cb316e5abff116e65277c7bb1&commit_b=360ae5f72680a37f487da2cde21947ec7f16d99b&stat=instructions%3Au

[petrochenkov]

@michaelwoerister
Updated table

setup	max-rss (average)	cpu-clock (average)
32-bit span, 2-bit tag	-0.73%	+0.16%
32-bit span, 1-bit tag	-0.70%	-1.05%
64-bit span, 1-bit tag	-0.66%	-0.77%

"32-bit span, 1-bit tag" still seems to be better

[michaelwoerister]

Alright, thanks for checking! 32 bits with 1 bit tag seems to be a good choice indeed.

Maybe encode() and decode() should be marked as #[inline] just so their body is available in downstream crates.

If you make the interner use an FxHashMap this is good to go :)

[tamird]

this comment has rotted.

[petrochenkov]

@michaelwoerister
Updated.

[michaelwoerister]

@bors r+

Thanks, @petrochenkov! Looking forward to seeing the results :)

[bors]

📌 Commit 52251cd has been approved by michaelwoerister

[bors]

⌛ Testing commit 52251cd with merge dcb4378...

[bors]

☀️ Test successful - status-appveyor, status-travis
Approved by: michaelwoerister
Pushing dcb4378 to master...

[alexcrichton]

Looks like this may have improved the memory of the tuple-stress benchmark by 5%!