Skip to content

Enable Miri to pass pointers through FFI - #129684

Merged
bors merged 1 commit into
rust-lang:masterfrom
Strophox:miri-pass-pointer-to-ffi
Aug 31, 2024
Merged

Enable Miri to pass pointers through FFI#129684
bors merged 1 commit into
rust-lang:masterfrom
Strophox:miri-pass-pointer-to-ffi

Conversation

@Strophox

@StrophoxStrophox commented Aug 28, 2024

Copy link
Copy Markdown
Contributor

Following #126787, the purpose of this PR is to now enable Miri to execute native calls that make use of pointers.

Simple example 
extern"C"{fnptr_printer(ptr:*muti32);}fnmain(){let ptr = &mut42as*muti32;unsafe{ptr_printer(ptr);}}
voidptr_printer(int*ptr) {
printf("printing pointer dereference from C: %d\n", *ptr);
}

should now show printing pointer dereference from C: 42.

Note that this PR does not yet implement any logic involved in updating Miri's "analysis" state (byte initialization, provenance) upon such a native call.

r? @RalfJung

@rustbotrustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Aug 28, 2024
@rustbot

rustbot commented Aug 28, 2024

Copy link
Copy Markdown
Collaborator

Some changes occurred to the CTFE / Miri engine

cc @rust-lang/miri

Some changes occurred to the CTFE / Miri engine

cc @rust-lang/miri

The Miri subtree was changed

cc @rust-lang/miri

@Strophox

Strophox commented Aug 28, 2024

Copy link
Copy Markdown
ContributorAuthor

Oh. Well that didn't work yet.

RalfJung
RalfJung reviewed Aug 28, 2024
View reviewed changes
Comment threadsrc/tools/miri/src/machine.rs Outdated
RalfJung
RalfJung reviewed Aug 28, 2024
View reviewed changes
Comment threadcompiler/rustc_middle/src/mir/interpret/allocation.rs Outdated
RalfJung
RalfJung reviewed Aug 28, 2024
View reviewed changes
Comment threadsrc/tools/miri/src/alloc_addresses/mod.rs Outdated
RalfJung
RalfJung reviewed Aug 29, 2024
View reviewed changes
Comment threadsrc/tools/miri/src/alloc_addresses/mod.rs Outdated
Comment threadsrc/tools/miri/src/alloc_addresses/mod.rs
Comment threadsrc/tools/miri/src/alloc_addresses/mod.rs Outdated
Comment threadsrc/tools/miri/src/machine.rs Outdated
Comment threadsrc/tools/miri/src/machine.rs Outdated
Comment threadsrc/tools/miri/src/machine.rs Outdated
Comment threadsrc/tools/miri/src/machine.rs Outdated
Comment threadsrc/tools/miri/src/shims/native_lib.rs Outdated
Comment threadsrc/tools/miri/tests/native-lib/pass/call_extern_c_fn.stdout
@@ -1 +1,2 @@
printing from C
printing pointer dereference from C: 42

@RalfJungRalfJungAug 29, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@rust-log-analyzer

This comment has been minimized.

Sign in to view
@Strophox

Strophox commented Aug 29, 2024

Copy link
Copy Markdown
ContributorAuthor

(I might quickly try to enhance+refactor the C pointer FFI tests in Miri now.)

RalfJung
RalfJung reviewed Aug 29, 2024
View reviewed changes
Comment threadsrc/tools/miri/src/alloc_addresses/mod.rs
RalfJung
RalfJung reviewed Aug 29, 2024
View reviewed changes
Comment threadsrc/tools/miri/src/alloc_addresses/mod.rs Outdated
Comment threadsrc/tools/miri/src/alloc_addresses/mod.rs
Comment threadsrc/tools/miri/src/machine.rs Outdated
RalfJung
RalfJung reviewed Aug 30, 2024
View reviewed changes
Comment threadsrc/tools/miri/src/alloc_addresses/mod.rs Outdated
RalfJung
RalfJung reviewed Aug 30, 2024
View reviewed changes
Comment threadsrc/tools/miri/src/alloc_addresses/mod.rs
@Strophox
Strophoxforce-pushed the miri-pass-pointer-to-ffi branch from dcdb101 to 6e23a4dCompareAugust 30, 2024 09:04
RalfJung
RalfJung reviewed Aug 30, 2024
View reviewed changes

@RalfJungRalfJung left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, thanks! Just got some nits regarding the tests.

Comment threadsrc/tools/miri/tests/native-lib/native-lib.map Outdated
Comment threadsrc/tools/miri/tests/native-lib/pass/ptr_read_access.rs Outdated
Comment threadsrc/tools/miri/tests/native-lib/pass/ptr_read_access.rs Outdated
Comment threadsrc/tools/miri/tests/native-lib/pass/scalar_arguments.rs Outdated
Comment threadsrc/tools/miri/tests/native-lib/ptr_read_access.c Outdated
Comment threadsrc/tools/miri/tests/native-lib/scalar_arguments.c Outdated
Comment threadsrc/tools/miri/tests/ui.rs Outdated
Comment threadsrc/tools/miri/tests/ui.rs Outdated
RalfJung
RalfJung reviewed Aug 30, 2024
View reviewed changes
Comment threadsrc/tools/miri/tests/native-lib/ptr_read_access.c Outdated
@RalfJung

RalfJung commented Aug 30, 2024

Copy link
Copy Markdown
Member

Please squash the commits, then we can land this. :-)

@Strophox@RalfJung
enable Miri to pass const pointers through FFI
7fde02e
Co-authored-by: Ralf Jung <post@ralfj.de>
@Strophox
Strophoxforce-pushed the miri-pass-pointer-to-ffi branch from 730bd97 to 7fde02eCompareAugust 30, 2024 14:12
@RalfJung

RalfJung commented Aug 31, 2024

Copy link
Copy Markdown
Member

Great, congrats on getting this finished!

@bors r+

@bors

bors commented Aug 31, 2024

Copy link
Copy Markdown
Collaborator

📌 Commit 7fde02e has been approved by RalfJung

It is now in the queue for this repository.

@borsbors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 31, 2024
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Aug 31, 2024
@matthiaskrgr
Rollup merge ofrust-lang#129684- Strophox:miri-pass-pointer-to-ffi,…
f995a63
… r=RalfJung
Enable Miri to pass pointers through FFI
Following rust-lang#126787, the purpose of this PR is to now enable Miri to execute native calls that make use of pointers.
> <details>
>
> <summary> Simple example </summary>
>
> ```rust
> extern "C" {
> fn ptr_printer(ptr: *mut i32);
> }
>
> fn main() {
> let ptr = &mut 42 as *mut i32;
> unsafe {
> ptr_printer(ptr);
> }
> }
> ```
> ```c
> void ptr_printer(int *ptr) {
> printf("printing pointer dereference from C: %d\n", *ptr);
> }
> ```
> should now show `printing pointer dereference from C: 42`.
>
> </details>
Note that this PR does not yet implement any logic involved in updating Miri's "analysis" state (byte initialization, provenance) upon such a native call.
r? `@RalfJung`
This was referenced Aug 31, 2024
Closed
Merged
bors added a commit to rust-lang-ci/rust that referenced this pull request Aug 31, 2024
@bors
Auto merge ofrust-lang#129831- matthiaskrgr:rollup-befq6zx, r=matth…
d37b074
…iaskrgr
Rollup of 11 pull requests
Successful merges:
- rust-lang#128523 (Add release notes for 1.81.0)
- rust-lang#129605 (Add missing `needs-llvm-components` directives for run-make tests that need target-specific codegen)
- rust-lang#129650 (Clean up `library/profiler_builtins/build.rs`)
- rust-lang#129651 (skip stage 0 target check if `BOOTSTRAP_SKIP_TARGET_SANITY` is set)
- rust-lang#129684 (Enable Miri to pass pointers through FFI)
- rust-lang#129762 (Update the `wasm-component-ld` binary dependency)
- rust-lang#129782 (couple more crash tests)
- rust-lang#129816 (tidy: say which feature gate has a stability issue mismatch)
- rust-lang#129818 (make the const-unstable-in-stable error more clear)
- rust-lang#129824 (Fix code examples buttons not appearing on click on mobile)
- rust-lang#129826 (library: Fix typo in `core::mem`)
r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Aug 31, 2024
@bors
Auto merge ofrust-lang#129831- matthiaskrgr:rollup-befq6zx, r=matth…
a7399ba
…iaskrgr
Rollup of 11 pull requests
Successful merges:
- rust-lang#128523 (Add release notes for 1.81.0)
- rust-lang#129605 (Add missing `needs-llvm-components` directives for run-make tests that need target-specific codegen)
- rust-lang#129650 (Clean up `library/profiler_builtins/build.rs`)
- rust-lang#129651 (skip stage 0 target check if `BOOTSTRAP_SKIP_TARGET_SANITY` is set)
- rust-lang#129684 (Enable Miri to pass pointers through FFI)
- rust-lang#129762 (Update the `wasm-component-ld` binary dependency)
- rust-lang#129782 (couple more crash tests)
- rust-lang#129816 (tidy: say which feature gate has a stability issue mismatch)
- rust-lang#129818 (make the const-unstable-in-stable error more clear)
- rust-lang#129824 (Fix code examples buttons not appearing on click on mobile)
- rust-lang#129826 (library: Fix typo in `core::mem`)
r? `@ghost`
`@rustbot` modify labels: rollup
@bors
bors merged commit a5fb8b9 into rust-lang:masterAug 31, 2024
@rustbotrustbot added this to the 1.82.0 milestone Aug 31, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Aug 31, 2024
@rust-timer
Unrolled build forrust-lang#129684
5632491
Rollup merge of rust-lang#129684 - Strophox:miri-pass-pointer-to-ffi, r=RalfJung
Enable Miri to pass pointers through FFI
Following rust-lang#126787, the purpose of this PR is to now enable Miri to execute native calls that make use of pointers.
> <details>
>
> <summary> Simple example </summary>
>
> ```rust
> extern "C" {
> fn ptr_printer(ptr: *mut i32);
> }
>
> fn main() {
> let ptr = &mut 42 as *mut i32;
> unsafe {
> ptr_printer(ptr);
> }
> }
> ```
> ```c
> void ptr_printer(int *ptr) {
> printf("printing pointer dereference from C: %d\n", *ptr);
> }
> ```
> should now show `printing pointer dereference from C: 42`.
>
> </details>
Note that this PR does not yet implement any logic involved in updating Miri's "analysis" state (byte initialization, provenance) upon such a native call.
r? ``@RalfJung``
@RalfJungRalfJung mentioned this pull request Sep 13, 2024
Open
7 tasks
@StrophoxStrophox mentioned this pull request Nov 19, 2024
Merged
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Dec 6, 2024
@matthiaskrgr
Rollup merge ofrust-lang#133211- Strophox:miri-correct-state-update…
76d0709
…-ffi, r=RalfJung
Extend Miri to correctly pass mutable pointers through FFI
Based off of rust-lang#129684, this PR further extends Miri to execute native calls that make use of pointers to *mutable* memory.
We adapt Miri's bookkeeping of internal state upon any FFI call that gives external code permission to mutate memory.
Native code may now possibly write and therefore initialize and change the pointer provenance of bytes it has access to: Such memory is assumed to be *initialized* afterwards and bytes are given *arbitrary (wildcard) provenance*. This enables programs that correctly use mutating FFI calls to run Miri without errors, at the cost of possibly missing Undefined Behaviour caused by incorrect usage of mutating FFI.
> <details>
>
> <summary> Simple example </summary>
>
> ```rust
> extern "C" {
> fn init_int(ptr: *mut i32);
> }
>
> fn main() {
> let mut x = std::mem::MaybeUninit::<i32>::uninit();
> let x = unsafe {
> init_int(x.as_mut_ptr());
> x.assume_init()
> };
>
> println!("C initialized my memory to: {x}");
> }
> ```
> ```c
> void init_int(int *ptr) {
> *ptr = 42;
> }
> ```
> should now show `C initialized my memory to: 42`.
>
> </details>
r? `@RalfJung`
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Dec 6, 2024
@matthiaskrgr
Rollup merge ofrust-lang#133211- Strophox:miri-correct-state-update…
576176d
…-ffi, r=RalfJung
Extend Miri to correctly pass mutable pointers through FFI
Based off of rust-lang#129684, this PR further extends Miri to execute native calls that make use of pointers to *mutable* memory.
We adapt Miri's bookkeeping of internal state upon any FFI call that gives external code permission to mutate memory.
Native code may now possibly write and therefore initialize and change the pointer provenance of bytes it has access to: Such memory is assumed to be *initialized* afterwards and bytes are given *arbitrary (wildcard) provenance*. This enables programs that correctly use mutating FFI calls to run Miri without errors, at the cost of possibly missing Undefined Behaviour caused by incorrect usage of mutating FFI.
> <details>
>
> <summary> Simple example </summary>
>
> ```rust
> extern "C" {
> fn init_int(ptr: *mut i32);
> }
>
> fn main() {
> let mut x = std::mem::MaybeUninit::<i32>::uninit();
> let x = unsafe {
> init_int(x.as_mut_ptr());
> x.assume_init()
> };
>
> println!("C initialized my memory to: {x}");
> }
> ```
> ```c
> void init_int(int *ptr) {
> *ptr = 42;
> }
> ```
> should now show `C initialized my memory to: 42`.
>
> </details>
r? ``@RalfJung``
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Dec 6, 2024
@rust-timer
Unrolled build forrust-lang#133211
e3b8092
Rollup merge of rust-lang#133211 - Strophox:miri-correct-state-update-ffi, r=RalfJung
Extend Miri to correctly pass mutable pointers through FFI
Based off of rust-lang#129684, this PR further extends Miri to execute native calls that make use of pointers to *mutable* memory.
We adapt Miri's bookkeeping of internal state upon any FFI call that gives external code permission to mutate memory.
Native code may now possibly write and therefore initialize and change the pointer provenance of bytes it has access to: Such memory is assumed to be *initialized* afterwards and bytes are given *arbitrary (wildcard) provenance*. This enables programs that correctly use mutating FFI calls to run Miri without errors, at the cost of possibly missing Undefined Behaviour caused by incorrect usage of mutating FFI.
> <details>
>
> <summary> Simple example </summary>
>
> ```rust
> extern "C" {
> fn init_int(ptr: *mut i32);
> }
>
> fn main() {
> let mut x = std::mem::MaybeUninit::<i32>::uninit();
> let x = unsafe {
> init_int(x.as_mut_ptr());
> x.assume_init()
> };
>
> println!("C initialized my memory to: {x}");
> }
> ```
> ```c
> void init_int(int *ptr) {
> *ptr = 42;
> }
> ```
> should now show `C initialized my memory to: 42`.
>
> </details>
r? ``@RalfJung``
github-actionsBot pushed a commit to rust-lang/miri that referenced this pull request Dec 7, 2024
@matthiaskrgr
Rollup merge of #133211 - Strophox:miri-correct-state-update-ffi, r=R…
64b7d3b
…alfJung
Extend Miri to correctly pass mutable pointers through FFI
Based off of rust-lang/rust#129684, this PR further extends Miri to execute native calls that make use of pointers to *mutable* memory.
We adapt Miri's bookkeeping of internal state upon any FFI call that gives external code permission to mutate memory.
Native code may now possibly write and therefore initialize and change the pointer provenance of bytes it has access to: Such memory is assumed to be *initialized* afterwards and bytes are given *arbitrary (wildcard) provenance*. This enables programs that correctly use mutating FFI calls to run Miri without errors, at the cost of possibly missing Undefined Behaviour caused by incorrect usage of mutating FFI.
> <details>
>
> <summary> Simple example </summary>
>
> ```rust
> extern "C" {
> fn init_int(ptr: *mut i32);
> }
>
> fn main() {
> let mut x = std::mem::MaybeUninit::<i32>::uninit();
> let x = unsafe {
> init_int(x.as_mut_ptr());
> x.assume_init()
> };
>
> println!("C initialized my memory to: {x}");
> }
> ```
> ```c
> void init_int(int *ptr) {
> *ptr = 42;
> }
> ```
> should now show `C initialized my memory to: 42`.
>
> </details>
r? ``@RalfJung``
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@RalfJungRalfJungRalfJung left review comments

Assignees

@RalfJungRalfJung

Labels

S-waiting-on-borsStatus: Waiting on bors to run and complete tests. Bors will change the label on completion.T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

Projects

None yet

Milestone

1.82.0

Development

Successfully merging this pull request may close these issues.

5 participants

@Strophox@rustbot@rust-log-analyzer@RalfJung@bors