-
Notifications
You must be signed in to change notification settings - Fork 12.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove the alloc_jemalloc
crate
#55238
Conversation
r? @estebank (rust_highfive has picked a reviewer for you, use r? to override) |
cc @sfackler |
This comment has been minimized.
This comment has been minimized.
Woo! There is one pretty significant difference between jemalloc 4 and 5. The new version depends on a background thread to clear out allocation caches, and the background thread defaults to off. Without it, we saw a very large increase in heap memory on a production service until we enabled background threads. See jemalloc/jemalloc#1128 for some background. While this doesn't matter for rustc at all, it will probably have an impact on long running services. We may want to fold the switch to the system allocator as the default into this change as well? |
This comment has been minimized.
This comment has been minimized.
As far as I understand, that switch is already part of this PR. |
7094bb6
to
94cb105
Compare
@sfackler ah yeah as @SimonSapin mentioned this PR is doing the switch as well as the update. Despite it looking like we enable the background thread by default in |
Those options compile in support for the background thread but it's still disabled at runtime by default. I don't think it would have much of an effect on rustc memory usage - my understanding is that it's used to clean up caches on idle threads and that active threads clean themselves up periodically when touching the allocator. There are a bunch of settings you can poke at to control how aggressively jemalloc caches if you want: http://jemalloc.net/jemalloc.3.html |
@bors r+ |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
The job Click to expand the log.
I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
This comment has been minimized.
This comment has been minimized.
// enable this by default on other platforms, so other platforms aren't handled | ||
// here yet. | ||
#[cfg(feature = "jemalloc-sys")] | ||
extern crate jemalloc_sys; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I get that on these platforms global_allocator
is not needed, but I still do wonder why this does not use it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The #[global_allocator]
attribute is actually incompatible with this due to the way linkage works and such. I can go into details if necessary, but the answer to your question at least is "it's required by the way we build rustc"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah damn, thanks for the explanation.
@bors: retry After many attempts locally I can't reproduce, so I'm curious if we can get another failure on bors... |
Just for clarity, does this change just affect |
@@ -2464,6 +2504,11 @@ impl OngoingCodegen { | |||
} | |||
} | |||
|
|||
// impl Drop for OngoingCodegen { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@alexcrichton what's this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah this is a mistake
Yes, |
@jonhoo this affects all crates, no crate will be compiled against jemalloc by default now. The compilers we distribute for OSX/Linux will, however, still enable jemalloc (but only the compiler itself) |
Here are the final perf results from the landing. Almost everything improved by at least 1%, and some were up to 15%. Pretty great! |
This is most likely from upgrading jemalloc from 4.5 to 5.1, isn’t it? |
Are there plans to do something about the memory usage regression? |
@lnicola the |
src/jemalloc is gone, remove its mention from COPYRIGHT The `src/jemalloc` submodule was removed in 61e8944 / rust-lang#55238.
src/jemalloc is gone, remove its mention from COPYRIGHT The `src/jemalloc` submodule was removed in 61e8944 / rust-lang#55238.
Changelog: Version 1.32.0 (2019-01-17) ========================== Language -------- #### 2018 edition - [You can now use the `?` operator in macro definitions.][56245] The `?` operator allows you to specify zero or one repetitions similar to the `*` and `+` operators. - [Module paths with no leading keyword like `super`, `self`, or `crate`, will now always resolve to the item (`enum`, `struct`, etc.) available in the module if present, before resolving to a external crate or an item the prelude.][56759] E.g. ```rust enum Color { Red, Green, Blue } use Color::*; ``` #### All editions - [You can now match against `PhantomData<T>` types.][55837] - [You can now match against literals in macros with the `literal` specifier.][56072] This will match against a literal of any type. E.g. `1`, `'A'`, `"Hello World"` - [Self can now be used as a constructor and pattern for unit and tuple structs.][56365] E.g. ```rust struct Point(i32, i32); impl Point { pub fn new(x: i32, y: i32) -> Self { Self(x, y) } pub fn is_origin(&self) -> bool { match self { Self(0, 0) => true, _ => false, } } } ``` - [Self can also now be used in type definitions.][56366] E.g. ```rust enum List<T> where Self: PartialOrd<Self> // can write `Self` instead of `List<T>` { Nil, Cons(T, Box<Self>) // likewise here } ``` - [You can now mark traits with `#[must_use]`.][55663] This provides a warning if a `impl Trait` or `dyn Trait` is returned and unused in the program. Compiler -------- - [The default allocator has changed from jemalloc to the default allocator on your system.][55238] The compiler itself on Linux & macOS will still use jemalloc, but programs compiled with it will use the system allocator. - [Added the `aarch64-pc-windows-msvc` target.][55702] Libraries --------- - [`PathBuf` now implements `FromStr`.][55148] - [`Box<[T]>` now implements `FromIterator<T>`.][55843] - [The `dbg!` macro has been stabilized.][56395] This macro enables you to easily debug expressions in your rust program. E.g. ```rust let a = 2; let b = dbg!(a * 2) + 1; // ^-- prints: [src/main.rs:4] a * 2 = 4 assert_eq!(b, 5); ``` The following APIs are now `const` functions and can be used in a `const` context. - [`Cell::as_ptr`] - [`UnsafeCell::get`] - [`char::is_ascii`] - [`iter::empty`] - [`ManuallyDrop::new`] - [`ManuallyDrop::into_inner`] - [`RangeInclusive::start`] - [`RangeInclusive::end`] - [`NonNull::as_ptr`] - [`slice::as_ptr`] - [`str::as_ptr`] - [`Duration::as_secs`] - [`Duration::subsec_millis`] - [`Duration::subsec_micros`] - [`Duration::subsec_nanos`] - [`CStr::as_ptr`] - [`Ipv4Addr::is_unspecified`] - [`Ipv6Addr::new`] - [`Ipv6Addr::octets`] Stabilized APIs --------------- - [`i8::to_be_bytes`] - [`i8::to_le_bytes`] - [`i8::to_ne_bytes`] - [`i8::from_be_bytes`] - [`i8::from_le_bytes`] - [`i8::from_ne_bytes`] - [`i16::to_be_bytes`] - [`i16::to_le_bytes`] - [`i16::to_ne_bytes`] - [`i16::from_be_bytes`] - [`i16::from_le_bytes`] - [`i16::from_ne_bytes`] - [`i32::to_be_bytes`] - [`i32::to_le_bytes`] - [`i32::to_ne_bytes`] - [`i32::from_be_bytes`] - [`i32::from_le_bytes`] - [`i32::from_ne_bytes`] - [`i64::to_be_bytes`] - [`i64::to_le_bytes`] - [`i64::to_ne_bytes`] - [`i64::from_be_bytes`] - [`i64::from_le_bytes`] - [`i64::from_ne_bytes`] - [`i128::to_be_bytes`] - [`i128::to_le_bytes`] - [`i128::to_ne_bytes`] - [`i128::from_be_bytes`] - [`i128::from_le_bytes`] - [`i128::from_ne_bytes`] - [`isize::to_be_bytes`] - [`isize::to_le_bytes`] - [`isize::to_ne_bytes`] - [`isize::from_be_bytes`] - [`isize::from_le_bytes`] - [`isize::from_ne_bytes`] - [`u8::to_be_bytes`] - [`u8::to_le_bytes`] - [`u8::to_ne_bytes`] - [`u8::from_be_bytes`] - [`u8::from_le_bytes`] - [`u8::from_ne_bytes`] - [`u16::to_be_bytes`] - [`u16::to_le_bytes`] - [`u16::to_ne_bytes`] - [`u16::from_be_bytes`] - [`u16::from_le_bytes`] - [`u16::from_ne_bytes`] - [`u32::to_be_bytes`] - [`u32::to_le_bytes`] - [`u32::to_ne_bytes`] - [`u32::from_be_bytes`] - [`u32::from_le_bytes`] - [`u32::from_ne_bytes`] - [`u64::to_be_bytes`] - [`u64::to_le_bytes`] - [`u64::to_ne_bytes`] - [`u64::from_be_bytes`] - [`u64::from_le_bytes`] - [`u64::from_ne_bytes`] - [`u128::to_be_bytes`] - [`u128::to_le_bytes`] - [`u128::to_ne_bytes`] - [`u128::from_be_bytes`] - [`u128::from_le_bytes`] - [`u128::from_ne_bytes`] - [`usize::to_be_bytes`] - [`usize::to_le_bytes`] - [`usize::to_ne_bytes`] - [`usize::from_be_bytes`] - [`usize::from_le_bytes`] - [`usize::from_ne_bytes`] Cargo ----- - [You can now run `cargo c` as an alias for `cargo check`.][cargo/6218] - [Usernames are now allowed in alt registry URLs.][cargo/6242] Misc ---- - [`libproc_macro` has been added to the `rust-src` distribution.][55280] Compatibility Notes ------------------- - [The argument types for AVX's `_mm256_stream_si256`, `_mm256_stream_pd`, `_mm256_stream_ps`][55610] have been changed from `*const` to `*mut` as the previous implementation was unsound. [55148]: rust-lang/rust#55148 [55238]: rust-lang/rust#55238 [55280]: rust-lang/rust#55280 [55610]: rust-lang/rust#55610 [55663]: rust-lang/rust#55663 [55702]: https://github.com/rust-lag/rust/pull/55702/ [55837]: rust-lang/rust#55837 [55843]: rust-lang/rust#55843 [56072]: rust-lang/rust#56072 [56245]: rust-lang/rust#56245 [56365]: https:/ttps://github.com/rust-lang/rust/pull/56395/ [56759]: rust-lang/rust#56759 [cargo/6218]: rust-lang/cargo#6218 [cargo/6242]: rust-lang/cargo#6242 [`CStr::as_ptr`]: https://doc.rust-`Duration::as_secs`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.as_secs [`Duration::subsec_micros`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.subsec_micros [`Duration::subsec_millis`]: https://doc.rust-lang.org/sct.Duration.html#method.subsec_nanos [`Ipv4Addr::is_unspecified`]: https://doc.rust-lang.org/std/net/struct.Ipv4Addr.html#method.is_unspecified [`Ipv6Addr::new`]: https://doc.rust-lang.org/std/net/struct.Ipv6Addr.html#method.new [`Ipv6Addr::octets`]: httpw`]: https://doc.rust-lang.org/std/mem/struct.ManuallyDrop.html#method.new [`NonNull::as_ptr`]: https://doc.rust-lang.org/std/ptr/struct.NonNull.html#method.as_ptr [`RangeInclusive::end`]: https://doc.rust-lang.org/std/ops/struct.RangeInclusive.html#method.end [`RangeInclusive::start`]: https://doc.rust-lang.org/std/ops/struct.RangeInclusive.html#method.start [`UnsafeCell::get`]: https://doc.rust-lang.org/std/cell/struct.UnsafeCell.html#method.get [`slice::as_ptr`]: https://doc.rust-lang.org/std/primitive.slice.html#method.as_ptr [`char::is_ascii`]: https://doc.rust-lang.org/std/primitive.char.html#method.is_ascii [`i128::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i128.html#method.from_be_bytes [`i128::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i128.html#method.from_le_bytes [`i128::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i128.html#method.from_ne_bytes [`i128::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i128.html#method.to_be_bytes [`i128::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i128.html#method.to_le_bytes [`i128::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i128.html#method.to_ne_bytes [`i16::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i16.html#method.from_be_bytes [`i16::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i16.html#method.from_le_bytes [`i16::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i16.html#method.from_ne_bytes [`i16::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i16.html#method.to_be_bytes [`i16::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i16.html#method.to_le_bytes [`i16::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i16.html#method.to_ne_bytes [`i32::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i32.html#method.from_be_bytes [`i32::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i32.html#method.from_le_bytes [`i32::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i32.html#method.from_ne_bytes [`i32::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i32.html#method.to_be_bytes [`i32::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i32.html#method.to_le_bytes [`i32::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i32.html#method.to_ne_bytes [`i64::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i64.html#method.from_be_bytes [`i64::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i64.html#method.from_le_bytes [`i64::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i64.html#method.from_ne_bytes [`i64::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i64.html#method.to_be_bytes [`i64::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i64.html#method.to_le_bytes [`i64::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i64.html#method.to_ne_bytes [`i8::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i8.html#method.from_be_bytes [`i8::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i8.html#method.from_le_bytes [`i8::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i8.html#method.from_ne_bytes [`i8::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i8.html#method.to_be_bytes [`i8::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i8.html#method.to_le_bytes [`i8::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i8.html#method.to_ne_bytes [`isize::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.isize.html#method.from_be_bytes [`isize::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.isize.html#method.from_le_bytes [`isize::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.isize.html#method.from_ne_bytes [`isize::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.isize.html#method.to_be_bytes [`isize::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.isize.html#method.to_le_bytes [`isize::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.isize.html#method.to_ne_bytes [`iter::empty`]: https://doc.rust-lang.org/std/iter/fn.empty.html [`str::as_ptr`]: https://doc.rust-lang.org/std/primitive.str.html#method.as_ptr [`u128::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u128.html#method.from_be_bytes [`u128::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u128.html#method.from_le_bytes [`u128::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u128.html#method.from_ne_bytes [`u128::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u128.html#method.to_be_bytes [`u128::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u128.html#method.to_le_bytes [`u128::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u128.html#method.to_ne_bytes [`u16::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u16.html#method.from_be_bytes [`u16::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u16.html#method.from_le_bytes [`u16::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u16.html#method.from_ne_bytes [`u16::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u16.html#method.to_be_bytes [`u16::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u16.html#method.to_le_bytes [`u16::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u16.html#method.to_ne_bytes [`u32::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u32.html#method.from_be_bytes [`u32::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u32.html#method.from_le_bytes [`u32::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u32.html#method.from_ne_bytes [`u32::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u32.html#method.to_be_bytes [`u32::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u32.html#method.to_le_bytes [`u32::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u32.html#method.to_ne_bytes [`u64::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u64.html#method.from_be_bytes [`u64::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u64.html#method.from_le_bytes [`u64::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u64.html#method.from_ne_bytes [`u64::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u64.html#method.to_be_bytes [`u64::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u64.html#method.to_le_bytes [`u64::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u64.html#method.to_ne_bytes [`u8::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.from_be_bytes [`u8::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.from_le_bytes [`u8::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.from_ne_bytes [`u8::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_be_bytes [`u8::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_le_bytes [`u8::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_ne_bytes [`usize::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.usize.html#method.from_be_bytes [`usize::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.usize.html#method.from_le_bytes [`usize::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.usize.html#method.from_ne_bytes [`usize::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.usize.html#method.to_be_bytes [`usize::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.usize.html#method.to_le_bytes [`usize::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.usize.html#method.to_ne_bytes
Changelog: Version 1.32.0 (2019-01-17) ========================== Language -------- #### 2018 edition - [You can now use the `?` operator in macro definitions.][56245] The `?` operator allows you to specify zero or one repetitions similar to the `*` and `+` operators. - [Module paths with no leading keyword like `super`, `self`, or `crate`, will now always resolve to the item (`enum`, `struct`, etc.) available in the module if present, before resolving to a external crate or an item the prelude.][56759] E.g. ```rust enum Color { Red, Green, Blue } use Color::*; ``` #### All editions - [You can now match against `PhantomData<T>` types.][55837] - [You can now match against literals in macros with the `literal` specifier.][56072] This will match against a literal of any type. E.g. `1`, `'A'`, `"Hello World"` - [Self can now be used as a constructor and pattern for unit and tuple structs.][56365] E.g. ```rust struct Point(i32, i32); impl Point { pub fn new(x: i32, y: i32) -> Self { Self(x, y) } pub fn is_origin(&self) -> bool { match self { Self(0, 0) => true, _ => false, } } } ``` - [Self can also now be used in type definitions.][56366] E.g. ```rust enum List<T> where Self: PartialOrd<Self> // can write `Self` instead of `List<T>` { Nil, Cons(T, Box<Self>) // likewise here } ``` - [You can now mark traits with `#[must_use]`.][55663] This provides a warning if a `impl Trait` or `dyn Trait` is returned and unused in the program. Compiler -------- - [The default allocator has changed from jemalloc to the default allocator on your system.][55238] The compiler itself on Linux & macOS will still use jemalloc, but programs compiled with it will use the system allocator. - [Added the `aarch64-pc-windows-msvc` target.][55702] Libraries --------- - [`PathBuf` now implements `FromStr`.][55148] - [`Box<[T]>` now implements `FromIterator<T>`.][55843] - [The `dbg!` macro has been stabilized.][56395] This macro enables you to easily debug expressions in your rust program. E.g. ```rust let a = 2; let b = dbg!(a * 2) + 1; // ^-- prints: [src/main.rs:4] a * 2 = 4 assert_eq!(b, 5); ``` The following APIs are now `const` functions and can be used in a `const` context. - [`Cell::as_ptr`] - [`UnsafeCell::get`] - [`char::is_ascii`] - [`iter::empty`] - [`ManuallyDrop::new`] - [`ManuallyDrop::into_inner`] - [`RangeInclusive::start`] - [`RangeInclusive::end`] - [`NonNull::as_ptr`] - [`slice::as_ptr`] - [`str::as_ptr`] - [`Duration::as_secs`] - [`Duration::subsec_millis`] - [`Duration::subsec_micros`] - [`Duration::subsec_nanos`] - [`CStr::as_ptr`] - [`Ipv4Addr::is_unspecified`] - [`Ipv6Addr::new`] - [`Ipv6Addr::octets`] Stabilized APIs --------------- - [`i8::to_be_bytes`] - [`i8::to_le_bytes`] - [`i8::to_ne_bytes`] - [`i8::from_be_bytes`] - [`i8::from_le_bytes`] - [`i8::from_ne_bytes`] - [`i16::to_be_bytes`] - [`i16::to_le_bytes`] - [`i16::to_ne_bytes`] - [`i16::from_be_bytes`] - [`i16::from_le_bytes`] - [`i16::from_ne_bytes`] - [`i32::to_be_bytes`] - [`i32::to_le_bytes`] - [`i32::to_ne_bytes`] - [`i32::from_be_bytes`] - [`i32::from_le_bytes`] - [`i32::from_ne_bytes`] - [`i64::to_be_bytes`] - [`i64::to_le_bytes`] - [`i64::to_ne_bytes`] - [`i64::from_be_bytes`] - [`i64::from_le_bytes`] - [`i64::from_ne_bytes`] - [`i128::to_be_bytes`] - [`i128::to_le_bytes`] - [`i128::to_ne_bytes`] - [`i128::from_be_bytes`] - [`i128::from_le_bytes`] - [`i128::from_ne_bytes`] - [`isize::to_be_bytes`] - [`isize::to_le_bytes`] - [`isize::to_ne_bytes`] - [`isize::from_be_bytes`] - [`isize::from_le_bytes`] - [`isize::from_ne_bytes`] - [`u8::to_be_bytes`] - [`u8::to_le_bytes`] - [`u8::to_ne_bytes`] - [`u8::from_be_bytes`] - [`u8::from_le_bytes`] - [`u8::from_ne_bytes`] - [`u16::to_be_bytes`] - [`u16::to_le_bytes`] - [`u16::to_ne_bytes`] - [`u16::from_be_bytes`] - [`u16::from_le_bytes`] - [`u16::from_ne_bytes`] - [`u32::to_be_bytes`] - [`u32::to_le_bytes`] - [`u32::to_ne_bytes`] - [`u32::from_be_bytes`] - [`u32::from_le_bytes`] - [`u32::from_ne_bytes`] - [`u64::to_be_bytes`] - [`u64::to_le_bytes`] - [`u64::to_ne_bytes`] - [`u64::from_be_bytes`] - [`u64::from_le_bytes`] - [`u64::from_ne_bytes`] - [`u128::to_be_bytes`] - [`u128::to_le_bytes`] - [`u128::to_ne_bytes`] - [`u128::from_be_bytes`] - [`u128::from_le_bytes`] - [`u128::from_ne_bytes`] - [`usize::to_be_bytes`] - [`usize::to_le_bytes`] - [`usize::to_ne_bytes`] - [`usize::from_be_bytes`] - [`usize::from_le_bytes`] - [`usize::from_ne_bytes`] Cargo ----- - [You can now run `cargo c` as an alias for `cargo check`.][cargo/6218] - [Usernames are now allowed in alt registry URLs.][cargo/6242] Misc ---- - [`libproc_macro` has been added to the `rust-src` distribution.][55280] Compatibility Notes ------------------- - [The argument types for AVX's `_mm256_stream_si256`, `_mm256_stream_pd`, `_mm256_stream_ps`][55610] have been changed from `*const` to `*mut` as the previous implementation was unsound. [55148]: rust-lang/rust#55148 [55238]: rust-lang/rust#55238 [55280]: rust-lang/rust#55280 [55610]: rust-lang/rust#55610 [55663]: rust-lang/rust#55663 [55702]: https://github.com/rust-lag/rust/pull/55702/ [55837]: rust-lang/rust#55837 [55843]: rust-lang/rust#55843 [56072]: rust-lang/rust#56072 [56245]: rust-lang/rust#56245 [56365]: https:/ttps://github.com/rust-lang/rust/pull/56395/ [56759]: rust-lang/rust#56759 [cargo/6218]: rust-lang/cargo#6218 [cargo/6242]: rust-lang/cargo#6242 [`CStr::as_ptr`]: https://doc.rust-`Duration::as_secs`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.as_secs [`Duration::subsec_micros`]: https://doc.rust-lang.org/std/time/struct.Duration.html#method.subsec_micros [`Duration::subsec_millis`]: https://doc.rust-lang.org/sct.Duration.html#method.subsec_nanos [`Ipv4Addr::is_unspecified`]: https://doc.rust-lang.org/std/net/struct.Ipv4Addr.html#method.is_unspecified [`Ipv6Addr::new`]: https://doc.rust-lang.org/std/net/struct.Ipv6Addr.html#method.new [`Ipv6Addr::octets`]: httpw`]: https://doc.rust-lang.org/std/mem/struct.ManuallyDrop.html#method.new [`NonNull::as_ptr`]: https://doc.rust-lang.org/std/ptr/struct.NonNull.html#method.as_ptr [`RangeInclusive::end`]: https://doc.rust-lang.org/std/ops/struct.RangeInclusive.html#method.end [`RangeInclusive::start`]: https://doc.rust-lang.org/std/ops/struct.RangeInclusive.html#method.start [`UnsafeCell::get`]: https://doc.rust-lang.org/std/cell/struct.UnsafeCell.html#method.get [`slice::as_ptr`]: https://doc.rust-lang.org/std/primitive.slice.html#method.as_ptr [`char::is_ascii`]: https://doc.rust-lang.org/std/primitive.char.html#method.is_ascii [`i128::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i128.html#method.from_be_bytes [`i128::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i128.html#method.from_le_bytes [`i128::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i128.html#method.from_ne_bytes [`i128::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i128.html#method.to_be_bytes [`i128::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i128.html#method.to_le_bytes [`i128::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i128.html#method.to_ne_bytes [`i16::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i16.html#method.from_be_bytes [`i16::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i16.html#method.from_le_bytes [`i16::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i16.html#method.from_ne_bytes [`i16::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i16.html#method.to_be_bytes [`i16::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i16.html#method.to_le_bytes [`i16::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i16.html#method.to_ne_bytes [`i32::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i32.html#method.from_be_bytes [`i32::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i32.html#method.from_le_bytes [`i32::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i32.html#method.from_ne_bytes [`i32::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i32.html#method.to_be_bytes [`i32::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i32.html#method.to_le_bytes [`i32::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i32.html#method.to_ne_bytes [`i64::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i64.html#method.from_be_bytes [`i64::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i64.html#method.from_le_bytes [`i64::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i64.html#method.from_ne_bytes [`i64::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i64.html#method.to_be_bytes [`i64::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i64.html#method.to_le_bytes [`i64::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i64.html#method.to_ne_bytes [`i8::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i8.html#method.from_be_bytes [`i8::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i8.html#method.from_le_bytes [`i8::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i8.html#method.from_ne_bytes [`i8::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i8.html#method.to_be_bytes [`i8::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i8.html#method.to_le_bytes [`i8::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.i8.html#method.to_ne_bytes [`isize::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.isize.html#method.from_be_bytes [`isize::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.isize.html#method.from_le_bytes [`isize::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.isize.html#method.from_ne_bytes [`isize::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.isize.html#method.to_be_bytes [`isize::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.isize.html#method.to_le_bytes [`isize::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.isize.html#method.to_ne_bytes [`iter::empty`]: https://doc.rust-lang.org/std/iter/fn.empty.html [`str::as_ptr`]: https://doc.rust-lang.org/std/primitive.str.html#method.as_ptr [`u128::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u128.html#method.from_be_bytes [`u128::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u128.html#method.from_le_bytes [`u128::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u128.html#method.from_ne_bytes [`u128::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u128.html#method.to_be_bytes [`u128::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u128.html#method.to_le_bytes [`u128::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u128.html#method.to_ne_bytes [`u16::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u16.html#method.from_be_bytes [`u16::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u16.html#method.from_le_bytes [`u16::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u16.html#method.from_ne_bytes [`u16::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u16.html#method.to_be_bytes [`u16::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u16.html#method.to_le_bytes [`u16::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u16.html#method.to_ne_bytes [`u32::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u32.html#method.from_be_bytes [`u32::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u32.html#method.from_le_bytes [`u32::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u32.html#method.from_ne_bytes [`u32::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u32.html#method.to_be_bytes [`u32::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u32.html#method.to_le_bytes [`u32::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u32.html#method.to_ne_bytes [`u64::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u64.html#method.from_be_bytes [`u64::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u64.html#method.from_le_bytes [`u64::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u64.html#method.from_ne_bytes [`u64::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u64.html#method.to_be_bytes [`u64::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u64.html#method.to_le_bytes [`u64::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u64.html#method.to_ne_bytes [`u8::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.from_be_bytes [`u8::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.from_le_bytes [`u8::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.from_ne_bytes [`u8::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_be_bytes [`u8::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_le_bytes [`u8::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_ne_bytes [`usize::from_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.usize.html#method.from_be_bytes [`usize::from_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.usize.html#method.from_le_bytes [`usize::from_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.usize.html#method.from_ne_bytes [`usize::to_be_bytes`]: https://doc.rust-lang.org/stable/std/primitive.usize.html#method.to_be_bytes [`usize::to_le_bytes`]: https://doc.rust-lang.org/stable/std/primitive.usize.html#method.to_le_bytes [`usize::to_ne_bytes`]: https://doc.rust-lang.org/stable/std/primitive.usize.html#method.to_ne_bytes
Tested with: rustc 1.32.0-nightly (25a42b2ce 2018-11-07) jemalloc was removed from the standard library recently: <rust-lang/rust#55238> As a result, we started seeing cases where the alignment requirements enforced on allocated regions are weaker than before. Some code ceased working because it erroneously relied on jemalloc's larger alignment values. This commit fixes this issue by introducing `AlignedVec`, a wrapper of `Vec`, which provides access to a portion of `Vec` that meets an alignment requirement required by SIMD operations, and by modifying code in question to use `AlignedVec` in place of `Vec`.
Tested with: rustc 1.32.0-nightly (25a42b2ce 2018-11-07) jemalloc was removed from the standard library recently: <rust-lang/rust#55238> As a result, we started seeing cases where the alignment requirements enforced on allocated regions are weaker than before. Some code ceased working because it erroneously relied on jemalloc's larger alignment values. This commit fixes this issue by introducing `AlignedVec`, a wrapper of `Vec`, which provides access to a portion of `Vec` that meets an alignment requirement required by SIMD operations, and by modifying code in question to use `AlignedVec` in place of `Vec`.
jemalloc was removed from the standard library recently: <rust-lang/rust#55238> Removed `extern crate alloc_system;` from `ngsloader` because this is the current default choice. `ngsengine` explicitly registers jemalloc (provided by the `jemallocator` crate) as the global allocator for optimal runtime performance.
switch `jemalloc-sys` back to `tikv-jemalloc-sys`, and update to 0.6.0 Some context: - we used to use jemalloc bindings from https://github.com/gnzlbg/jemallocator, since rust-lang#55238 - that crate was abandoned, picked up as a fork in https://github.com/tikv/jemallocator, so we switched to that in rust-lang#83152. - then they were able to publish to the original `jemalloc-sys` bindings crate, and `jemalloc-sys` and `tikv-jemalloc-sys` became the same thing -- so I switched back to the OG crate in rust-lang#96790 - they're now having publishing problems again: I've been waiting for tikv/jemallocator#96 for the `jemalloc-sys` 0.6.0 update for a few months, but `tikv-jemalloc-sys` is already updated to 0.6.0. A perf run showed some improvements, so this PR switches back to `tikv-jemalloc-sys` to update to 0.6.0. r? ghost
switch `jemalloc-sys` back to `tikv-jemalloc-sys`, and update to 0.6.0 Some context: - we used to use jemalloc bindings from https://github.com/gnzlbg/jemallocator, since rust-lang#55238 - that crate was abandoned, picked up as a fork in https://github.com/tikv/jemallocator, so we switched to that in rust-lang#83152. - then they were able to publish to the original `jemalloc-sys` bindings crate, and `jemalloc-sys` and `tikv-jemalloc-sys` became the same thing -- so I switched back to the OG crate in rust-lang#96790 - they're now having publishing problems again: I've been waiting for tikv/jemallocator#96 for the `jemalloc-sys` 0.6.0 update for a few months, but `tikv-jemalloc-sys` is already updated to 0.6.0. A perf run showed some improvements, so this PR switches back to `tikv-jemalloc-sys` to update to 0.6.0.
This commit removes the
alloc_jemalloc
crate from the standard library and all related configuration. We will no longer be shipping this unstable crate. Rationale for this is provided on #36963 and the many linked issues, but I can inline rationale here if desired!We currently rely on jemalloc for increased perf in the Rust compiler, however. This perf run shows that if we switch to glibc 2.23's allocator that it's slower than jemalloc across many benchmarks. This perf run, however, shows that if we use
jemalloc-sys
from crates.io then rustc actually gets faster across all benchmarks! (presumably because it has a more recent version of jemalloc than our submodule).As a result, it's expected that this doesn't regress any code (as it's just removing an unstable crate) and it should actually improve rustc performance because it updates jemalloc.
Closes #36963