New behavior of --feature + --package combination

[matklad]

Docs: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#package-features

A tracking issue for #5353 and #5351. Historically, --feature and related flags applied to the current package, and not to the package, specified via --package. This is a bug, but fixing it could break someone's code, so currently new behavior exists under -Z package-features opt-in. We need to check if it breaks code in the wild to see what we should do next.

[matklad]

#5362 is I think the manifestation of the old odd behavior.

[matklad]

Announcement on irlo: https://internals.rust-lang.org/t/help-us-test-the-breaking-bug-fix-to-cargo-features/7317

[matklad]

I think #4942 is also this issue.

[matklad]

And #4753 as well

[matklad]

This broke Servo unfortunately, so we probably would need to back this out, at least in this, maximally aggressive, form.

However, I wonder what we plan to do with feature-selection problems in general?

I assume that we want to eventually fix the main problem with features. Currently, features are global per compilation, and that means that features bleed from dev-deps to normal deps and that cargo build -p foo and cargo build -p foo -p bar build the foo package differently.

Fixing that, however, would be a breaking change: suddenly, Cargo would select a smaller set of features :-(

But, if we do fix that, then behavior of the combination of --package and --features would necessary change as well. In other words, this PR is sort of a subset of that larger issue.

The core question I think is do we feel comfortable with eventually changing the set of activated features? If we are, then I think we can modify current fix to not flatly error-out, but to ignore the features argument for several packages, printing a warning like "this was accepted in previous versions of Cargo and activated features in surprising way, and not it is ignored".

[pravic]

$ cargo +stable test --all --features xyz

Used to work. And expectation was "Enable the specified set of futures for crates that support it".

$ cargo +nightly test --all --features xyz
error: cannot specify features for more than one package

Now we see this (thanks to CI to catch it soon and not after Rust release) and it does not matter whether all of workspace members have xyz or not, error is the same.

How come that the original issue (cargo --package foo --feature feat) has affected builds without --package? #5390 wasn't a good idea without a compatibility explanation (what is it? how to fix that mystical error? how it will work in future?). That error must have contained at least the issue number (since it is a breaking future and it is in nightly (i.e. unstable) anyway).

[matklad]

And expectation was "Enable the specified set of futures for crates that support it".

Not that this expectation is wrong. What actually happens is that the feature is enabled only for the package in the current working directory. In other words, it is irrelevant whether all members have the xyz feature, because this applies only to a single member, and not the one that is specified via the -p argument.

That error must have contained at least the issue number

That was definitely an oversight on my part :(

[matklad]

Here's a small example which demonstrates the difference in behavior: https://github.com/matklad/features-behavior

[matklad]

has affected builds without --package?

--all is a shorthand for --package member1 --package member2 --package member3 etc, and suffers the same problem: --feature flag does not apply to the selected packages. I've extended the example at https://github.com/matklad/features-behavior to show how this affects --all.

[SimonSapin]

Breaking changes to stable functionality are not acceptable. Please revert and find a way to make this opt-in.

However weird and subjectively broken the old behavior is, it has been around for more than long enough that many large projects have built work-arounds that came to rely on it. It’s easy to say this in hindsight, but I think it shouldn’t be surprising that a change like this is definitely gonna break some builds.

#5390, the PR that enable this change by default, says:

So far, the feedback on https://internals.rust-lang.org/t/help-us-test-the-breaking-bug-fix-to-cargo-features/7317 has been positive

At the time of that writing, that forum thread was 2 days old and had responses from three individuals, one of them saying their build is broken. How is this "positive" enough?

But even if that discussion had received community-wide attention, breakage like this without opt-in is still unacceptable IMO. The projects most likely to be affected are those with complex build system that involve many crates. They may or may not be involved in the day-to-day development of Rust and Cargo.

---

Now if we are going to make a change around feature selection, in addition to being opt-in (possibly through edition = "2018" in the root crate or workspace?) I think we should spent a lot more time and involve more people to consider what the new behavior should be. In particular, fixing #4463. And possibly separating the notions of “Choosing which of the several 'root' artifacts in a workspace (each with its dependency graph and feature selection)” and “Choosing a subset of one dependency graph”.

[gnzlbg]

Because of this cargo bug stdsimd has been silently not testing some feature combinations (rust-lang/stdarch#569). That this produces no warnings whatsoever worries me.

[gnzlbg]

Breaking changes to stable functionality are not acceptable.

This is incorrect. Breaking changes to stable functionality are acceptable if the functionality is broken, otherwise we couldn't ever fix any bugs if the behavior of the fix could be observed by current crates.

The question that we have to resolve here is whether this functionality is broken or not. If it isn't broken, you are correct in that we cannot fix it. But if it is broken, we can fix it. Depending on the impact of the fix, we might have to warn first for a sufficiently long time before applying the fix, but fixing it is possible and does not require a new edition, although we could use one to implement the fix.

Currently, when I have a workspace with a package foo with a feature bar, executing:

cargo test --features=bar -p foo

compiles foo without the feature bar. I personally think this behavior is broken, and that those wanting this behavior should just write cargo test -p foo without --features=bar.

[yihuang]

For me, it would be enough to have something like:

cargo build -C subcrate --features ...

Which behaves like:

cd subcrate
cargo build --features ...
cd ..

[sfackler]

cargo build --features ... --manifest-path subcrate/Cargo.toml.

[ehuss]

I am making a proposal to change the behavior of -Zpackage-features in #8074. I would appreciate any feedback from anyone following this issue.

The intent is to cover more use cases that people have expressed interest in, and to push this towards stabilization. The new behavior is summarized in the docs. Compared to the old -Zpackage-features behavior, it changes:

• You can specify features for multiple packages at once. Not all packages need to define every feature, Cargo only enables matching features.
• Adds --features member_name/feature_name syntax for setting features on specific members.

I think it is possible to stabilize the new behavior as-is except for the one backwards-incompatible change of -p foo --features … inside a member's directory. I can't think of a good transition plan for that, so I think a solution is to make the new behavior opt-in. We are planning an opt-in mechanism for the new feature resolver, so I'm thinking it could just ride on that same option (probably a flag specified in the [workspace] definition).

[jonhoo]

@ehuss I like that proposal a lot!