cargo doc in workspaces includes removed crates' documentation if their previously built documentation is still in target/doc

[robojumper]

Problem
Building documentation for a workspace, removing a crate from that workspace, and building documentation again keeps documentation for that removed crate accessible in the sidebar and search index.

Steps
I have a repository to reproduce this at https://github.com/robojumper/cargo-doc-workspace-issue.

PowerShell console transcript:

PS D:\Rust> cargo --version
cargo 1.35.0-nightly (0e35bd8af 2019-03-13)
PS D:\Rust> rustc --version
rustc 1.35.0-nightly (0576ac109 2019-03-24)
PS D:\Rust> git clone https://github.com/robojumper/cargo-doc-workspace-issue.git
Cloning into 'cargo-doc-workspace-issue'...
remote: Enumerating objects: 17, done.
remote: Counting objects: 100% (17/17), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 17 (delta 1), reused 17 (delta 1), pack-reused 0
Unpacking objects: 100% (17/17), done.
PS D:\Rust> cd .\cargo-doc-workspace-issue\
PS D:\Rust\cargo-doc-workspace-issue> git checkout 26190d
Note: checking out '26190d'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 26190dc... Initial Commit
PS D:\Rust\cargo-doc-workspace-issue> tree /F
Auflistung der Ordnerpfade für Volume DATA
Volumeseriennummer : 9EE8-64D7
D:.
│   .gitignore
│   Cargo.toml
│
├───liba
│   │   Cargo.toml
│   │
│   └───src
│           lib.rs
│
├───libb
│   │   Cargo.toml
│   │
│   └───src
│           lib.rs
│
└───src
        lib.rs

PS D:\Rust\cargo-doc-workspace-issue> cargo doc --all --no-deps
 Documenting liba v0.1.0 (D:\Rust\cargo-doc-workspace-issue\liba)
 Documenting libb v0.1.0 (D:\Rust\cargo-doc-workspace-issue\libb)
    Finished dev [unoptimized + debuginfo] target(s) in 2.39s
PS D:\Rust\cargo-doc-workspace-issue> git checkout dde0a0
Previous HEAD position was 26190dc... Initial Commit
HEAD is now at dde0a01... Remove libb
PS D:\Rust\cargo-doc-workspace-issue> cargo doc --all --no-deps
    Finished dev [unoptimized + debuginfo] target(s) in 0.04s
PS D:\Rust\cargo-doc-workspace-issue> tree /F
Auflistung der Ordnerpfade für Volume DATA
Volumeseriennummer : 9EE8-64D7
D:.
│   .gitignore
│   Cargo.lock
│   Cargo.toml
│
├───liba
│   │   Cargo.toml
│   │
│   └───src
│           lib.rs
│
└───target
    │   .rustc_info.json
    │
    ├───debug
    │   │   .cargo-lock
    │   │
    │   ├───.fingerprint
    │   │   ├───liba-41cbe19eb88fc838
    │   │   │       doc-lib-liba-41cbe19eb88fc838
    │   │   │       doc-lib-liba-41cbe19eb88fc838.json
    │   │   │
    │   │   └───libb-c20586c9d60cc169
    │   │           doc-lib-libb-c20586c9d60cc169
    │   │           doc-lib-libb-c20586c9d60cc169.json
    │   │
    │   ├───build
    │   ├───deps
    │   ├───examples
    │   ├───incremental
    └───doc
        │   .lock
        │   aliases.js
        │   brush.svg
        │   COPYRIGHT.txt
        │   dark.css
        │   favicon.ico
        │   FiraSans-LICENSE.txt
        │   FiraSans-Medium.woff
        │   FiraSans-Regular.woff
        │   LICENSE-APACHE.txt
        │   LICENSE-MIT.txt
        │   light.css
        │   main.js
        │   normalize.css
        │   noscript.css
        │   rust-logo.png
        │   rustdoc.css
        │   search-index.js
        │   settings.css
        │   settings.html
        │   settings.js
        │   source-files.js
        │   source-script.js
        │   SourceCodePro-LICENSE.txt
        │   SourceCodePro-Regular.woff
        │   SourceCodePro-Semibold.woff
        │   SourceSerifPro-Bold.ttf.woff
        │   SourceSerifPro-It.ttf.woff
        │   SourceSerifPro-LICENSE.txt
        │   SourceSerifPro-Regular.ttf.woff
        │   storage.js
        │   theme.js
        │   wheel.svg
        │
        ├───implementors
        │   └───core
        │       └───marker
        │               trait.Send.js
        │               trait.Sync.js
        │
        ├───liba
        │       all.html
        │       index.html
        │       sidebar-items.js
        │       struct.StructA.html
        │
        ├───libb
        │       all.html
        │       index.html
        │       sidebar-items.js
        │       struct.StructB.html
        │
        └───src
            ├───liba
            │       lib.rs.html
            │
            └───libb
                    lib.rs.html

Expected behavior
Opening target/doc/liba/index.html shows only items from liba in the sidebar and search index.

Actual behavior
Opening target/doc/liba/index.html shows liba, libb in the sidebar; libb and libb::StructB are still in the search index and have doc pages.

Notes
I haven't made any changes to liba after removing libb, but even with changes to liba, its pages still reference libb the same way.

This popped up in the master docs for rust-analyzer -- ra_ide_api_light has been removed yet is accessible through all other doc pages and the search index. EDIT: Looks like matklad cleared the Travis cache, so that's a workaround (as is the equivalent cargo clean).

[jyn514]

This also caches the documentation across toolchain versions, which is incorrect - cargo +stable doc && cargo +nightly doc --open should not reuse the same generated docs.

[ehuss]

@jyn514 That should be partially fixed a few days ago by #8640. (It doesn't protect retroactively, so it will only really work once it hits stable.)

[branpk]

Speaking frankly - this issue gave me a bad impression of cargo doc when I was new to Rust. Everything else in the cargo build system behaves well when building incrementally, and there never seem to be remnants across builds (which is a common problem with other build systems). This is a noticeable counterexample which new users are likely to run into.

The reason it's likely for new users to run into it is that the default cargo doc command includes dependencies, which I think is usually undesirable. After noticing that, the next command a user will run is cargo doc --no-deps, which then seems to have no effect because of this issue. (I'm assuming this is the same issue since it's caused by leftover crates in target/doc.)