Validate miri against the HIR const evaluator by oli-obk · Pull Request #45002 · rust-lang/rust

oli-obk · 2017-10-03T14:45:17Z

r? @eddyb

cc @alexcrichton @arielb1 @RalfJung

The interesting parts are the last few functions in librustc_const_eval/eval.rs

We warn if miri produces an error while HIR const eval does not.
We warn if miri produces a value that does not match the value produced by HIR const eval
if miri succeeds and HIR const eval fails, nothing is emitted, but we still return the HIR error
if both error, nothing is emitted and the HIR const eval error is returned

So there are no actual changes, except that miri is forced to produce the same values as the old const eval.

This does not touch the const evaluator in trans at all. That will come in a future PR.
This does not cause any code to compile that didn't compile before. That will also come in the future

It would be great if someone could start a crater run if travis passes

make force_allocation handle packed ByValPair

fix for latest nightly

previously miri had a check for const fn and other cases that CTFE requires. Instead the function call is completely processed inside the machine. This allows CTFE to have full control over what is called and miri to not have useless CTFE-checks in normal mode.

Split up miri into the librustc_mir and bin parts

Process untagged unions

oli-obk · 2017-12-14T13:30:31Z

At this point I don't know where the increase in size is coming from. Miri is adding nothing measurable to librustc and like 2MB to librustc_mir. I don't see how the other crates' size increases can come from miri.

It's a 23MB increase, which still is 12%. All I can think of now is that it severly degraded the zippability of the crates, because the unzipped crates seem to not have increased by that much.

kennytm · 2017-12-14T13:49:22Z

@oli-obk

zippability of the crates

Since the tarballs are actually uploaded we could download them and check locally.

	Before (`8954b16`)	After (`a9b425b`)
Download URL	(URL)	(URL)
File size	194,457,881	223,798,237
After `gunzip`	664,856,576	804,350,976

It doesn't look like related to zippability, but an actual increase in binary size.

Edit:

Comparing top largest files in the tarball (sorted by "after"):

File	Before	After
rls-preview/bin/rls	18,232,376	97,001,776
cargo/bin/cargo	17,335,552	57,398,408
rustc/lib/librustc_llvm-*.so	56,813,280	56,813,112
rust-std-/lib/rustlib//lib/librustc_llvm-*.so	56,813,280	56,813,112
rust-std-/lib/rustlib//lib/librustc_asan-*.rlib	26,420,558	26,420,544
rustc/lib/librustc-*.so	23,119,928	23,468,920
rust-std-/lib/rustlib//lib/librustc-*.so	23,119,928	23,468,920
rust-std-/lib/rustlib//lib/libcore-*.rlib	18,266,456	18,267,880
rust-std-/lib/rustlib//lib/librustc_tsan-*.rlib	17,178,772	17,178,750
rust-std-/lib/rustlib//lib/libstd-*.rlib	16,608,602	16,626,360
rustfmt-preview/bin/rustfmt	3,257,872	16,353,720

Why did RLS and Cargo becomes so much larger? They contribute almost all of the size increase.

oli-obk · 2017-12-14T14:16:46Z

folder	before	after
rustfmt-preview	9.0	23.0
cargo	17.6	51.8
rls-preview	18.3	84.9
rust-docs	149.6	149.6
rustc	171.6	159.6
rust-std	303.1	299.4

soo... uhm... I've been looking at the wrong thing the entire time

eddyb · 2017-12-14T14:20:30Z

Did we change the situations in which we generate MIR for libraries, or the MIR itself? Are we compiling some new dependencies as rlibs and thus duplicating them in the binaries somehow?

oli-obk · 2017-12-14T14:21:21Z

 if let Some(target_linker) = self.build.linker(target) {
 cargo.env("RUSTC_TARGET_LINKER", target_linker);
 }
+ cargo.env("RUSTC_DEBUGINFO", self.config.rust_debuginfo.to_string())


found the issue ~~(I think)~~

kennytm · 2017-12-14T14:39:59Z

Specifically checking the size increase of RLS:

	Before	After
URL	(URL)	(URL)
Size	4,641,992	14,975,372
After `unxz`	18,311,680	97,080,832
`rls-preview/bin/rls` binary size	18,232,376	97,001,776

Comparing the sections with `llvm-readelf -sections rls-preview/bin/rls`:

It's definitely the debug symbols. All the extra size is occupied by debug symbols.

oli-obk · 2017-12-14T14:43:21Z

While it's not fun to debug the tools without debug info, I can just activate it when needed. (confirmed locally that the binary size of the tools is back to what it's supposed to be)

Launching miri

3...
2...
1...
@bors r=eddyb

bors · 2017-12-14T14:43:21Z

📌 Commit 7a2bff7 has been approved by eddyb

bors · 2017-12-14T14:43:49Z

💡 This pull request was already approved, no need to approve it again.

There's another pull request that is currently being tested, blocking this pull request: Use PathBuf instead of String where applicable #46335

bors · 2017-12-14T14:43:49Z

📌 Commit 7a2bff7 has been approved by eddyb

bors · 2017-12-14T15:34:25Z

💡 This pull request was already approved, no need to approve it again.

There's another pull request that is currently being tested, blocking this pull request: Use PathBuf instead of String where applicable #46335

bors · 2017-12-14T15:34:26Z

📌 Commit 7a2bff7 has been approved by eddyb

bors · 2017-12-14T15:37:46Z

⌛ Testing commit 7a2bff7 with merge 2974104...

@eddyb

Validate miri against the HIR const evaluator r? @eddyb cc @alexcrichton @arielb1 @RalfJung The interesting parts are the last few functions in `librustc_const_eval/eval.rs` * We warn if miri produces an error while HIR const eval does not. * We warn if miri produces a value that does not match the value produced by HIR const eval * if miri succeeds and HIR const eval fails, nothing is emitted, but we still return the HIR error * if both error, nothing is emitted and the HIR const eval error is returned So there are no actual changes, except that miri is forced to produce the same values as the old const eval. * This does **not** touch the const evaluator in trans at all. That will come in a future PR. * This does **not** cause any code to compile that didn't compile before. That will also come in the future It would be great if someone could start a crater run if travis passes

bors · 2017-12-14T18:22:50Z

☀️ Test successful - status-appveyor, status-travis
Approved by: eddyb
Pushing 2974104 to master...

vn971 · 2021-01-22T08:18:30Z

Hi, is there any reason why .editorconfig was removed? See first changed file in acdf83f
CC @oli-obk

oli-obk · 2021-01-22T08:40:45Z

I don't remember... sorry

oli-obkand others added 30 commits July 31, 2017 12:41

Merge pull requestrust-lang#274from RalfJung/packed2
7c6befe
make force_allocation handle packed ByValPair

fix for latest nightly
69fd22f

Merge pull requestrust-lang#277from RalfJung/nightly
6d3c88c
fix for latest nightly

Split up miri into the librustc_mir and bin parts
11e30a6

Implement the first machine function
36502a3

Split "new" miri crate into modules
dc6e877

Move more non-CTFE operations to the Machine
4a4640a

Rename const_eval::Evaluator to CompileTimeFunctionEvaluator
960dca1

Move the global_item function to the EvalContext
728bb87

Fix doc text
e15d374

Move a method used everywhere out of intrinsic.rs
df7c42b

Update env_logger in order to share more dependencies
45b7cfd

Move all intrinsics out of interpret and fail CTFE on intrinsic calls
f16b9e2

Adjust travis to the directory changes
4383fc2

Update Cargo.toml
7ed706d

Allow machines to create new memory kinds
adfea61

Move tls code to its own file
23d351d

Remove leftover unused file
54ecdf1

Remove redundant members field
3d82b66

Properly process ptr_op for const eval
8de8580

Address comments
d05784f

Fix running the rustc test suite
8b81f54

Add a better error message than syscall 318 not found
393216d

Process untagged unions
79f38bf

Build the rustc_tests binary on travis
f19d77e

Merge pull requestrust-lang#268from oli-obk/upstream
7700bd0
Split up miri into the librustc_mir and bin parts

Add a test for untagged unions
f8c61da

Merge pull requestrust-lang#278from solson/unions
cf318b1
Process untagged unions

Produce backtraces for miri internals
29a3c4d

oli-obk commented Dec 14, 2017
View reviewed changes

Do not produce debuginfo for tools
7a2bff7

kennytm added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Dec 14, 2017

bors merged commit 7a2bff7 into rust-lang:masterDec 14, 2017

This was referenced Dec 14, 2017
Add sync module to rustc_data_structures #46709
Merged
Move collector to librustc_mir::monomorphize #45525
Merged

alexcrichton mentioned this pull request Jan 8, 2018
Significantly slower --emit=metadata #47267
Closed

kennytm mentioned this pull request Jan 27, 2018
Cargo update; and support converting *.xz to *.gz when promoting a release. rust-lang/rust-central-station#31
Merged

tmandry mentioned this pull request Jan 29, 2018
Say this was merged into rustc rust-lang/miri#362
Closed

solson mentioned this pull request Mar 20, 2018
upstream to rustc rust-lang/miri#105
Closed

vn971 mentioned this pull request Jan 22, 2021
Add .editorconfig rusqlite/rusqlite#595
Open

vn971 mentioned this pull request Jan 22, 2021
Add .editorconfig #81260
Merged

Uh oh!

Uh oh!

Conversation

oli-obk commented Oct 3, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oli-obk commented Dec 14, 2017

Uh oh!

kennytm commented Dec 14, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oli-obk commented Dec 14, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eddyb commented Dec 14, 2017

Uh oh!

oli-obkDec 14, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kennytm commented Dec 14, 2017• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oli-obk commented Dec 14, 2017• edited by shepmaster Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bors commented Dec 14, 2017

Uh oh!

bors commented Dec 14, 2017

Uh oh!

bors commented Dec 14, 2017

Uh oh!

bors commented Dec 14, 2017

Uh oh!

bors commented Dec 14, 2017

Uh oh!

bors commented Dec 14, 2017

Uh oh!

bors commented Dec 14, 2017

Uh oh!

vn971 commented Jan 22, 2021

Uh oh!

oli-obk commented Jan 22, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

15 participants

oli-obk commented Oct 3, 2017•
edited
Loading

kennytm commented Dec 14, 2017•
edited
Loading

oli-obk commented Dec 14, 2017•
edited
Loading

oli-obkDec 14, 2017•
edited
Loading

kennytm commented Dec 14, 2017•
edited
Loading

oli-obk commented Dec 14, 2017•
edited by shepmaster
Loading