Extend Miri to correctly pass mutable pointers through FFI #133211

Strophox · 2024-11-19T14:55:19Z

Based off of #129684, this PR further extends Miri to execute native calls that make use of pointers to mutable memory.
We adapt Miri's bookkeeping of internal state upon any FFI call that gives external code permission to mutate memory.

Native code may now possibly write and therefore initialize and change the pointer provenance of bytes it has access to: Such memory is assumed to be initialized afterwards and bytes are given arbitrary (wildcard) provenance. This enables programs that correctly use mutating FFI calls to run Miri without errors, at the cost of possibly missing Undefined Behaviour caused by incorrect usage of mutating FFI.

Simple example

extern "C" {
  fn init_int(ptr: *mut i32);
}

fn main() {
  let mut x = std::mem::MaybeUninit::<i32>::uninit();
  let x = unsafe {
    init_int(x.as_mut_ptr());
    x.assume_init()
  };

  println!("C initialized my memory to: {x}");
}

void init_int(int *ptr) {
  *ptr = 42;
}

should now show C initialized my memory to: 42.

r? @RalfJung

rustbot · 2024-11-19T14:55:29Z

Some changes occurred to the CTFE / Miri interpreter

cc @rust-lang/miri, @rust-lang/wg-const-eval

Some changes occurred to the CTFE machinery

cc @rust-lang/wg-const-eval

The Miri subtree was changed

cc @rust-lang/miri

Some changes occurred to the CTFE / Miri interpreter

cc @rust-lang/miri

Strophox · 2024-11-19T14:56:18Z

There are still some TODOs I need to get to, but this should make review easier.

compiler/rustc_const_eval/src/interpret/memory.rs

compiler/rustc_middle/src/mir/interpret/allocation.rs

src/tools/miri/src/shims/native_lib.rs

src/tools/miri/tests/native-lib/pass/ptr_write_access.rs

compiler/rustc_middle/src/mir/interpret/allocation/provenance_map.rs

compiler/rustc_const_eval/src/interpret/memory.rs

compiler/rustc_middle/src/mir/interpret/allocation.rs

compiler/rustc_middle/src/mir/interpret/allocation/provenance_map.rs

src/tools/miri/tests/native-lib/pass/ptr_write_access.rs

src/tools/miri/tests/native-lib/ptr_write_access.c

src/tools/miri/tests/native-lib/pass/ptr_write_access.rs

compiler/rustc_const_eval/src/interpret/memory.rs

src/tools/miri/src/alloc_addresses/mod.rs

src/tools/miri/tests/native-lib/pass/ptr_write_access.rs

RalfJung · 2024-11-30T10:43:10Z

@rustbot author

RalfJung · 2024-12-05T21:25:18Z

Okay that should be it. :)
Please squash this into a few (or a single) commits -- using git rebase --keep-base ... to avoid a diff.

Co-authored-by: Ralf Jung <[email protected]>

RalfJung · 2024-12-05T21:58:29Z

Thanks, this is good to go now!

@bors r+

bors · 2024-12-05T21:58:31Z

📌 Commit 712ceab has been approved by RalfJung

It is now in the queue for this repository.

…-ffi, r=RalfJung Extend Miri to correctly pass mutable pointers through FFI Based off of rust-lang#129684, this PR further extends Miri to execute native calls that make use of pointers to *mutable* memory. We adapt Miri's bookkeeping of internal state upon any FFI call that gives external code permission to mutate memory. Native code may now possibly write and therefore initialize and change the pointer provenance of bytes it has access to: Such memory is assumed to be *initialized* afterwards and bytes are given *arbitrary (wildcard) provenance*. This enables programs that correctly use mutating FFI calls to run Miri without errors, at the cost of possibly missing Undefined Behaviour caused by incorrect usage of mutating FFI. > <details> > > <summary> Simple example </summary> > > ```rust > extern "C" { > fn init_int(ptr: *mut i32); > } > > fn main() { > let mut x = std::mem::MaybeUninit::<i32>::uninit(); > let x = unsafe { > init_int(x.as_mut_ptr()); > x.assume_init() > }; > > println!("C initialized my memory to: {x}"); > } > ``` > ```c > void init_int(int *ptr) { > *ptr = 42; > } > ``` > should now show `C initialized my memory to: 42`. > > </details> r? `@RalfJung`

…iaskrgr Rollup of 5 pull requests Successful merges: - rust-lang#130777 (rust_for_linux: -Zreg-struct-return commandline flag for X86 (rust-lang#116973)) - rust-lang#133211 (Extend Miri to correctly pass mutable pointers through FFI) - rust-lang#133294 (crashes: more tests) - rust-lang#133790 (Improve documentation for Vec::extend_from_within) - rust-lang#133930 (rustbook: update to use new mdbook-trpl package from The Book) r? `@ghost` `@rustbot` modify labels: rollup

…iaskrgr Rollup of 5 pull requests Successful merges: - rust-lang#130777 (rust_for_linux: -Zreg-struct-return commandline flag for X86 (rust-lang#116973)) - rust-lang#133211 (Extend Miri to correctly pass mutable pointers through FFI) - rust-lang#133790 (Improve documentation for Vec::extend_from_within) - rust-lang#133930 (rustbook: update to use new mdbook-trpl package from The Book) - rust-lang#133931 (Only allow PassMode::Direct for aggregates on wasm when using the C ABI) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of rust-lang#133211 - Strophox:miri-correct-state-update-ffi, r=RalfJung Extend Miri to correctly pass mutable pointers through FFI Based off of rust-lang#129684, this PR further extends Miri to execute native calls that make use of pointers to *mutable* memory. We adapt Miri's bookkeeping of internal state upon any FFI call that gives external code permission to mutate memory. Native code may now possibly write and therefore initialize and change the pointer provenance of bytes it has access to: Such memory is assumed to be *initialized* afterwards and bytes are given *arbitrary (wildcard) provenance*. This enables programs that correctly use mutating FFI calls to run Miri without errors, at the cost of possibly missing Undefined Behaviour caused by incorrect usage of mutating FFI. > <details> > > <summary> Simple example </summary> > > ```rust > extern "C" { > fn init_int(ptr: *mut i32); > } > > fn main() { > let mut x = std::mem::MaybeUninit::<i32>::uninit(); > let x = unsafe { > init_int(x.as_mut_ptr()); > x.assume_init() > }; > > println!("C initialized my memory to: {x}"); > } > ``` > ```c > void init_int(int *ptr) { > *ptr = 42; > } > ``` > should now show `C initialized my memory to: 42`. > > </details> r? ``@RalfJung``

rustbot assigned RalfJung Nov 19, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Nov 19, 2024

This comment has been minimized.

Sign in to view

RalfJung reviewed Nov 21, 2024

View reviewed changes

This comment has been minimized.

Sign in to view

RalfJung reviewed Nov 24, 2024

View reviewed changes

compiler/rustc_middle/src/mir/interpret/allocation/provenance_map.rs Outdated Show resolved Hide resolved

RalfJung reviewed Nov 26, 2024

View reviewed changes

compiler/rustc_const_eval/src/interpret/memory.rs Show resolved Hide resolved