Add hooks for Miri panic unwinding

[Aaron1011]

This commits adds in some additional hooks to allow Miri to properly
handle panic unwinding. None of this should have any impact on CTFE mode

This supports rust-lang/miri#693

[rust-highfive]

r? @estebank

(rust_highfive has picked a reviewer for you, use r? to override)

[Aaron1011]

r? @RalfJung

[Aaron1011]

@RalfJung: Are there any changes that you'd like me to make?

[RalfJung]

Sorry, I didn't yet have the chance to look at this. It's on my list though!

[RalfJung]

I am a bit surprised by the implementation strategy here. Logically speaking, what happens with unwinding is that every function has two "return continuations", as in, two "return addresses" that it might jump to when execution is done: the successful one for normal completion, and the "unwind" address for when unwinding is happening.

So, what I'd expect is that StackPopCleanup::Goto stores to addresses to continue at, one for return and one for unwind. Then when a stack frame gets popped, whether that is a "return" or an "unwind" pop decides which of the two addresses we jump to.

I think with that approach you can also keep using the normal interpreter loop in run, instead of having to run your own loop (which, as @bjorn3 mentioned, won't work well for Priroda). (EDIT: Ah, turns out that loop was for the box_me_up call, so that's different.)

Is there a particular reason you chose the approach you did (which I am still trying to understand)?

[RalfJung]

Also, this is super cool! I didn't expect unwind support for Miri to happen any time soon, so I am very happy to see you tackle this. :)

[Aaron1011]

@RalfJung: My main goal was to make the minimum amount of changes to rustc, while leaving the core logic in Miri.

[RalfJung]

If you want to keep that kind of information out of the engine, make StackPopCleanup generic over Machine, add a StackPopCleanup assoc type to Machine and add a Machine variant to StackPopCleanup that contains a single value of Machine::StackPopCleanup type. Then miri can fill in its own enum there.

TBH that sounds like overkill to me. The action that happens on a pop is so simple (just take the unwind continuation instead of the return continuation), I don't think this warrants the mental overhead of a machine hook. I mean, it's literally just let target_block = if unwind { unwind_block } else { return_block }; self.goto(target_block).

[oli-obk]

I have no personal preference. I was mainly aiming to explain how to achieve @Aaron1011 's target

[rust-highfive]

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:02752a56:start=1559097167387587572,finish=1559097168209689654,duration=822102082
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout
Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---
[00:04:47] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:04:47] tidy error: /checkout/src/librustc_mir/interpret/terminator.rs:10: line longer than 100 chars
[00:04:47] tidy error: /checkout/src/librustc_mir/interpret/eval_context.rs:56: trailing whitespace
[00:04:47] tidy error: /checkout/src/librustc_mir/interpret/eval_context.rs:574: line longer than 100 chars
[00:04:47] tidy error: /checkout/src/librustc_mir/interpret/eval_context.rs:578: line longer than 100 chars
[00:04:47] tidy error: /checkout/src/librustc_mir/interpret/eval_context.rs:664: line longer than 100 chars
[00:04:47] tidy error: /checkout/src/librustc_mir/interpret/eval_context.rs:684: line longer than 100 chars
[00:04:47] tidy error: /checkout/src/librustc_mir/interpret/eval_context.rs:697: trailing whitespace
[00:04:52] some tidy checks failed
[00:04:52] [00:04:52] [00:04:52] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:52] [00:04:52] [00:04:52] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:52] Build completed unsuccessfully in 0:01:12
[00:04:52] Build completed unsuccessfully in 0:01:12
[00:04:52] make: *** [tidy] Error 1
[00:04:52] Makefile:67: recipe for target 'tidy' failed
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:27789af2
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Wed May 29 02:37:51 UTC 2019
---
travis_time:end:1f038874:start=1559097471905737640,finish=1559097471910094757,duration=4357117
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0bfb89c9
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:08298745
travis_time:start:08298745
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:0c2d8d9f
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[Aaron1011]

@RalfJung: I've made the changes you requested

[bors]

☔ The latest upstream changes (presumably #61278) made this pull request unmergeable. Please resolve the merge conflicts.

[bors]

☔ The latest upstream changes (presumably #61467) made this pull request unmergeable. Please resolve the merge conflicts.

[bors]

⌛ Testing commit b4545a4 with merge 62bbf07...

[rust-highfive]

The job dist-x86_64-msvc of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-11-12T15:57:45.4259967Z do so (now or later) by using -b with the checkout command again. Example:
2019-11-12T15:57:45.4260049Z 2019-11-12T15:57:45.4260102Z git checkout -b <new-branch-name>
2019-11-12T15:57:45.4260144Z 2019-11-12T15:57:45.4260382Z HEAD is now at 62bbf0704 Auto merge of #60026 - Aaron1011:feature/miri-unwind, r=RalfJung,oli-obk
2019-11-12T15:57:45.4587050Z ##[section]Starting: Decide whether to run this job
2019-11-12T15:57:45.4682757Z ==============================================================================
2019-11-12T15:57:45.4682863Z Task : Bash
2019-11-12T15:57:45.4682935Z Description : Run a Bash script on macOS, Linux, or Windows
---
2019-11-12T15:57:47.2473778Z ANT_HOME=C:\ProgramData\chocolatey\lib\ant\apache-ant-1.10.5
2019-11-12T15:57:47.2474272Z APPDATA=C:\Users\VssAdministrator\AppData\Roaming
2019-11-12T15:57:47.2474571Z AZURE_EXTENSION_DIR=C:\Program Files\Common Files\AzureCliExtensionDirectory
2019-11-12T15:57:47.2474791Z AZURE_HTTP_USER_AGENT=VSTS_d439fc94-e01f-4249-b63e-d8392bc0247c_build_10_0
2019-11-12T15:57:47.2474997Z Add hooks for Miri panic unwinding
2019-11-12T15:57:47.2475385Z BOOST_ROOT_1_69_0=C:\Program Files\Boost\1.69.0
2019-11-12T15:57:47.2475580Z BUILD_ARTIFACTSTAGINGDIRECTORY=D:\a\1\a
2019-11-12T15:57:47.2475756Z BUILD_BINARIESDIRECTORY=D:\a\1\b
2019-11-12T15:57:47.2475949Z BUILD_BUILDID=13383
---
2019-11-12T15:57:47.2511980Z TMP=/tmp
2019-11-12T15:57:47.2512164Z TOOLSTATE_ISSUES_API_URL=https://api.github.com/repos/rust-lang/rust/issues
2019-11-12T15:57:47.2512368Z TOOLSTATE_PUBLISH=1
2019-11-12T15:57:47.2512575Z TOOLSTATE_REPO=https://github.com/rust-lang-nursery/rust-toolstate
2019-11-12T15:57:47.2512781Z This commits adds in some additional hooks to allow Miri to properly
2019-11-12T15:57:47.2512998Z This supports https://github.com/rust-lang/miri/pull/693
2019-11-12T15:57:47.2513382Z USERDOMAIN=fv-az425
2019-11-12T15:57:47.2513569Z USERDOMAIN_ROAMINGPROFILE=fv-az425
2019-11-12T15:57:47.2513759Z USERNAME=VssAdministrator
2019-11-12T15:57:47.2514026Z USERPROFILE=C:\Users\VssAdministrator
---
2019-11-12T15:57:47.2515064Z WINDIR=C:\windows
2019-11-12T15:57:47.2515244Z WIX=C:\Program Files (x86)\WiX Toolset v3.11\
2019-11-12T15:57:47.2515439Z _=/usr/bin/printenv
2019-11-12T15:57:47.2515629Z agent.jobstatus=Succeeded
2019-11-12T15:57:47.2515822Z handle panic unwinding. None of this should have any impact on CTFE mode
2019-11-12T15:57:47.2516159Z disk usage:
2019-11-12T15:57:47.2516356Z Filesystem Size Used Avail Use% Mounted on
2019-11-12T15:57:47.2516545Z C:/Program Files/Git 256G 140G 116G 55% /
2019-11-12T15:57:47.2516762Z D: 14G 2.0G 13G 14% /d
---
2019-11-12T18:01:17.8612448Z [RUSTC-TIMING] shell_escape test:false 0.673
2019-11-12T18:01:17.8901365Z Compiling hex v0.4.0
2019-11-12T18:01:20.8940093Z [RUSTC-TIMING] hex test:false 3.999
2019-11-12T18:01:20.8977180Z Compiling glob v0.3.0
2019-11-12T18:01:21.3370489Z memory allocation of 4294967304 bytes failed[RUSTC-TIMING] hex test:false 3.438
2019-11-12T18:01:21.3472541Z error: could not compile `hex`.
2019-11-12T18:01:22.8915944Z [RUSTC-TIMING] glob test:false 1.984
2019-11-12T18:01:22.9032050Z error: build failed
2019-11-12T18:01:22.9081592Z command did not execute successfully: "D:\\a\\1\\s\\build\\x86_64-pc-windows-msvc\\stage0\\bin\\cargo.exe" "build" "-Zconfig-profile" "--target" "x86_64-pc-windows-msvc" "-Zbinary-dep-depinfo" "-j" "2" "--release" "--locked" "--color" "always" "--manifest-path" "D:\\a\\1\\s\\src/tools/cargo\\Cargo.toml" "--features" "rustc-workspace-hack/all-static" "--message-format" "json-render-diagnostics"
2019-11-12T18:01:22.9082199Z expected success, got: exit code: 101
2019-11-12T18:01:22.9082199Z expected success, got: exit code: 101
2019-11-12T18:01:23.0062715Z failed to run: D:\a\1\s\build\bootstrap\debug\bootstrap dist
2019-11-12T18:01:23.0062904Z Build completed unsuccessfully in 1:50:42
2019-11-12T18:01:23.0854580Z == clock drift check ==
2019-11-12T18:01:23.1965228Z local time: Tue Nov 12 18:01:23 CUT 2019
2019-11-12T18:01:23.5238939Z network time: Tue, 12 Nov 2019 18:01:23 GMT
2019-11-12T18:01:23.5255910Z == end clock drift check ==
2019-11-12T18:01:23.5954489Z 2019-11-12T18:01:24.0526927Z ##[error]Bash exited with code '1'.
2019-11-12T18:01:24.0975944Z ##[section]Starting: Checkout
2019-11-12T18:01:24.2030621Z ==============================================================================
2019-11-12T18:01:24.2031026Z Task : Get sources
2019-11-12T18:01:24.2031125Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[bors]

💔 Test failed - checks-azure

[Aaron1011]

memory allocation of 4294967304 bytes failed[RUSTC-TIMING] hex test:false 3.438

It looks like dist-x86_64-msvc somehow ran out of memory while building cargo.

@RalfJung Can you re-try this?

[bjorn3]

4GB memory allocation during compilation of hex is a lot.

[Aaron1011]

Note that Windows dist-x86_64-msvc succeed on the previous try with this commit (a different job timed out, so the build failed). This is either a spurious failure, or some weird interaction with something that was merged in the meantime.

[RalfJung]

Well let's try one last time.

@bors retry rollup=never

[bors]

⌛ Testing commit b4545a4 with merge a333eed...

[bors]

☀️ Test successful - checks-azure
Approved by: RalfJung,oli-obk
Pushing a333eed to master...