Avoid `Operand::Copy` with `&mut T` by jonas-schievink · Pull Request #72093 · rust-lang/rust

jonas-schievink · 2020-05-10T22:41:48Z

This is generally unsound to do, as the copied type is assumed to implement
Copy.

Closes#46420

rust-highfive · 2020-05-10T22:41:51Z

r? @petrochenkov

(rust_highfive has picked a reviewer for you, use r? to override)

petrochenkov · 2020-05-11T00:07:27Z

r? @RalfJung

RalfJung · 2020-05-11T07:27:14Z

This code is entirely unfamiliar to me, sorry... no idea why @petrochenkov thought I might be a good reviewer here.
r? @oli-obk maybe?

oli-obk · 2020-05-11T10:10:17Z

Please add a mir-opt test showing this

oli-obk · 2020-05-11T10:15:05Z

I think if we're enforcing this invariant, we should have a mir pass that validates the various (I guess we have more) invariants that we want to be upheld. We could have a flag that runs this MirPass after every other MirPass to ensure that all of our transformations are sound (enough). We don't want to do this in production, as that'll regress perf, but we could run the pass once before going to llvm.

That said, I am not sure what actual effects Operand::Copy has vs Operand::Move except for maybe borrowck? I always thought that post-borrowck we could normalize everything to Operand::Copy (or do my preferred change which is replace Copy and Move by a single variant and add a bool flag that defines which variant we're using here).

cc @rust-lang/wg-mir-opt

RalfJung · 2020-05-11T10:20:20Z

There are ideas about making a Move actually replace the source operand with uninitialized memory (@eddyb suggested this AFAIK -- there is an issue about this somewhere but I cannot find it right now). But so far it is unclear which optimizations this actually enables, and having a read perform mutation like that could also cause plenty of problems on its own.

EDIT: Here is the issue: rust-lang/unsafe-code-guidelines#188.

jonas-schievink · 2020-05-11T11:21:20Z

Having Operand::Move uninitialize the moved-from place would certainly help out with generator optimizations and also makes NRVO simpler

RalfJung · 2020-05-11T11:23:43Z

@jonas-schievink please post details in the issue I linked. So far, any time someone claimed that, a closer look revealed deinit-on-Move wasn't actually sufficient and we rather need something like #61849 to also rule out re-initialization via raw pointers.

jonas-schievink · 2020-05-11T11:24:34Z

Ah, yes, that one is what I meant

RalfJung · 2020-05-11T11:26:34Z

#61849 is very different from rust-lang/unsafe-code-guidelines#188.

#61849 is basically free(ptr); ptr = malloc(...);. So, even if there are aliases elsewhere they are now definitely dead, and even the observed address can change.

Deinit-on-Move (rust-lang/unsafe-code-guidelines#188) changes the content of the storage, but on its own it doesn't do anything about aliases (though maybe Stacked Borrows could be used to achieve something like that, further research is needed here). And it certainly does nothing wrt. equality of observed addresses.

bors · 2020-05-21T10:38:00Z

☔ The latest upstream changes (presumably #72205) made this pull request unmergeable. Please resolve the merge conflicts.

jonas-schievink · 2020-05-23T22:57:09Z

I've added a small validation pass and a -Zvalidate-mir flag to run it after each optimization pass.

RalfJung · 2020-05-23T23:00:46Z

I've added a small validation pass and a -Zvalidate-mir flag to run it after each optimization pass.

That's awesome, I about about 100 ideas for things this could check.^^ Just talked about that with @eddyb earlier today, in fact.

Can we run that pass at least once, say at the beginning and end of mir_optimized (in particular, before and after generator transform), even without the -Z flag when debug assertions are enabled? Then we could cover the entire test suite on the debug CI builders.

jonas-schievink · 2020-05-23T23:23:10Z

I ran it on the testsuite already and found nothing. Right now it only checks 2 invariants though (Copy-ness of Operand::Copy, and self-assignments). Which other ones did you have in mind? Is the conversation you had public?

RalfJung · 2020-05-23T23:41:59Z

No that was PMs on Zulip.

The main other check is: for every assignment, make sure the type of LHS and RHS match. (They are not always fully equal though, just "compatible enough"). We could also check that there is only one Return in the entire function. We could check that the operand of a Call is of type FnDef or FnPtr. And so on.

RalfJung · 2020-05-24T07:59:36Z

I ran it on the testsuite already and found nothing.

Good to know that this is true right now, but I'd prefer to make sure it stays that way. :) A check after every pass is probably overkill for that, but some check should IMO be done by default when debug assertions are enabled.

oli-obk · 2020-05-25T16:20:50Z

I agree with running this check once when debug assertions are enabled. Maybe at the end of optimized_mir so we at least know that the final thing we give to codegen is sane?

r=me with that

jonas-schievink · 2020-05-25T20:51:39Z

@bors r=oli-obk rollup=never

bors · 2020-05-25T20:51:40Z

📌 Commit fe1753a has been approved by oli-obk

bors · 2020-05-26T07:20:58Z

⌛ Testing commit fe1753a with merge cb272d5...

bors · 2020-05-26T10:42:33Z

☀️ Test successful - checks-azure
Approved by: oli-obk
Pushing cb272d5 to master...

RalfJung · 2020-05-30T21:18:01Z

 body.phase = mir_phase;
+
+ if mir_phase == MirPhase::Optimized {
+ validate::Validator { when: format!("end of phase {:?}", mir_phase) }


So we run it here even without debug assertions? Isn't that expensive?

The pass does very little work, so it's unlikely to be an issue

The pass does very little work
That is something I want to change.^^ #72796 does some non-trivial type folding.

@jonas-schievink

…ewjasper MIR sanity check: validate types on assignment This expands the MIR validation added by @jonas-schievink in rust-lang#72093 to also check that on an assignment, the types of both sides match. Cc @eddyb @oli-obk

@jonas-schievink

…ewjasper MIR sanity check: validate types on assignment This expands the MIR validation added by @jonas-schievink in rust-lang#72093 to also check that on an assignment, the types of both sides match. Cc @eddyb @oli-obk

@jonas-schievink

…ewjasper MIR sanity check: validate types on assignment This expands the MIR validation added by @jonas-schievink in rust-lang#72093 to also check that on an assignment, the types of both sides match. Cc @eddyb @oli-obk

rust-highfive assigned petrochenkov May 10, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 10, 2020

This comment has been minimized.
Sign in to view

rust-highfive assigned RalfJung and unassigned petrochenkov May 11, 2020

rust-highfive assigned oli-obk and unassigned RalfJung May 11, 2020

oli-obk reviewed May 11, 2020
View reviewed changes

Comment threadsrc/librustc_mir/transform/instcombine.rs Outdated

oli-obk reviewed May 11, 2020
View reviewed changes

Comment threadsrc/librustc_mir/shim.rs Outdated
oli-obkMay 11, 2020
Copy link
Copy Markdown
Contributor
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add a mir-opt test showing this

Elinvynia added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 20, 2020

jonas-schievink added 3 commits May 25, 2020 21:44

Avoid Operand::Copy with &mut T
4ca6262

Add a small MIR validation pass
e04318e

Always validate MIR after optimizing
fe1753a

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 25, 2020

bors added the merged-by-bors This PR was explicitly merged by bors. label May 26, 2020

bors merged commit cb272d5 into rust-lang:masterMay 26, 2020

RalfJung reviewed May 30, 2020
View reviewed changes

jonas-schievink deleted the unmut branch May 30, 2020 21:32

RalfJung mentioned this pull request May 30, 2020
MIR sanity check: validate types on assignment #72796
Merged

jonas-schievink mentioned this pull request May 30, 2020
InstCombine can incorrectly deinitialize locals that are later used #72797
Closed

cuviper mentioned this pull request Jun 2, 2020
InstCombine: Don't optimize &mut *x into x#72820
Merged

simonvandel mentioned this pull request Sep 16, 2020
MIR-opt: Generalize &(*_1) optimization to mutable references #76802
Closed

matthiaskrgr mentioned this pull request Aug 9, 2023
ICE: unboxed closure but only with mir-opt-level=0#114665
Closed

Uh oh!

Uh oh!

Conversation

jonas-schievink commented May 10, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rust-highfive commented May 10, 2020

Uh oh!

This comment has been minimized.

petrochenkov commented May 11, 2020

Uh oh!

RalfJung commented May 11, 2020

Uh oh!

Uh oh!

oli-obkMay 11, 2020

Choose a reason for hiding this comment

Uh oh!

oli-obk commented May 11, 2020

Uh oh!

RalfJung commented May 11, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jonas-schievink commented May 11, 2020

Uh oh!

RalfJung commented May 11, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jonas-schievink commented May 11, 2020

Uh oh!

RalfJung commented May 11, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bors commented May 21, 2020

Uh oh!

jonas-schievink commented May 23, 2020

Uh oh!

RalfJung commented May 23, 2020

Uh oh!

jonas-schievink commented May 23, 2020

Uh oh!

RalfJung commented May 23, 2020

Uh oh!

RalfJung commented May 24, 2020

Uh oh!

oli-obk commented May 25, 2020• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jonas-schievink commented May 25, 2020

Uh oh!

bors commented May 25, 2020

Uh oh!

bors commented May 26, 2020

Uh oh!

bors commented May 26, 2020

Uh oh!

RalfJungMay 30, 2020

Choose a reason for hiding this comment

Uh oh!

jonas-schievinkMay 30, 2020

Choose a reason for hiding this comment

Uh oh!

RalfJungMay 31, 2020

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

jonas-schievink commented May 10, 2020•
edited
Loading

RalfJung commented May 11, 2020•
edited
Loading

RalfJung commented May 11, 2020•
edited
Loading

RalfJung commented May 11, 2020•
edited
Loading

oli-obk commented May 25, 2020•
edited
Loading