Bootstrap build fails: found crate serde_derive compiled by an incompatible version of rustc
#125578
Comments
rustbot
added
the
needs-triage
onur-ozkan
added
T-bootstrap
So it doesn't happen after |
|
I removed |
|
Presumably bootstrap used the outdated artifacts instead of generating them with the new compiler (#125478). |
|
I did expect cargo to clean them up when it sees that the rustc version changed. Maybe something is broken with that? |
Yes. I think cargo hashes the version and channel, but they are unchanged for this beta update: 1.79.0, beta. The fact that it's beta.1 vs beta.6 should still usually trigger a rebuild though I would think, it should just use the same filenames... Cc @epage @weihanglo |
onur-ozkan
added
the
T-cargo
|
The root cause is likely to be that Cargo's cache for I can think of a heavy-hammer solution: We can additionally compute the digest of rustc executable. Anyone has a lightweight alternative? |
|
Is A cheaper alternative would be to key on the size and mtime of the binary. |
For small programs it was at least. A single rustc invocation used to be at least 100ms (for some people almost 300ms) with the rustup wrapper overhead included. Rustup optimizations have since reduced this by a significant amount, but it is still about 4x slower with the rustup wrapper than without for me. |
I think there is more nuance here. Normally cargo can distinguish different This will likely be a problem with every bootstrap bump moving forward. I'm trying to think if this is going to be a more serious problem for other non-rustup users of cargo. I looked at the linux I can think of two possible solutions: A slightly hacky solution would be to add the file size to cargo's hash.1 We can be pretty confident that the size will change, and that is cheap to acquire. Another possibility is to have Footnotes
|
|
I think that we should somehow update the timestamp in the source builds, e.g. based on the time of the latest commit at the time of packaging, rather than hardcode the same time for everything. I'll take a look at it. |
|
I have had to wipe my build dir like 8 times while cycling through different PRs, and I've accidentally rebuilt LLVM about 5 times. |
Right.. I thought it's worth bootstrap to have workaround for this and created #125911 PR. So people don't have to waste their time finding the actual problem and checking issues/zulip. |
|
Yeaaah. Redownloading all the bootstrap compilers and then restarting the build isn't exactly optimal, it just happened to be faster, for me, than "try to think about what the actual issue is". Not really sure why the LLVM got rebuilt though... ...also, 11. |
|
FWIW |
…bertlarsan68 delete bootstrap build before switching to bumped rustc Technically, wiping bootstrap builds can increase the build time. But in practice, trying to manually resolve post-bump issues and even accidentally removing the entire build directory will result in a much greater loss of time. After all, the bootstrap build process is not a particularly lengthy operation. Workaround for rust-lang#125578
…bertlarsan68 delete bootstrap build before switching to bumped rustc Technically, wiping bootstrap builds can increase the build time. But in practice, trying to manually resolve post-bump issues and even accidentally removing the entire build directory will result in a much greater loss of time. After all, the bootstrap build process is not a particularly lengthy operation. Workaround for rust-lang#125578
Rollup merge of rust-lang#125911 - onur-ozkan:wipe-broken-cache, r=albertlarsan68 delete bootstrap build before switching to bumped rustc Technically, wiping bootstrap builds can increase the build time. But in practice, trying to manually resolve post-bump issues and even accidentally removing the entire build directory will result in a much greater loss of time. After all, the bootstrap build process is not a particularly lengthy operation. Workaround for rust-lang#125578
…-ozkan Make mtime of reproducible tarballs dependent on git commit Since rust-lang#123246, our tarballs should be fully reproducible. That means that the mtime of all files and directories in the tarballs is set to the date of the first Rust commit (from 2006). However, this is causing some mtime invalidation issues (rust-lang#125578 (comment)). Ideally, we would like to keep the mtime reproducible, but still update it with new versions of Rust. That's what this PR does. It modifies the tarball installer bootstrap invocation so that if the current rustc directory is managed by git, we will set the UTC timestamp of the latest commit as the mtime for all files in the archive. This means that the archive should be still fully reproducible from a given commit SHA, but it will also be changed with new beta bumps and `download-rustc` versions. Note that only files are set to this mtime, directories are still set to the year 2006, because the `tar` library used by `rust-installer` doesn't allow us to selectively override mtime for directories (or at least I haven't found it). We could work around that by doing all the mtime modifications in bootstrap, but that would require more changes. I think/hope that just modifying the file mtimes should be enough. It should at least fix cargo `rustc` mtime invalidation. Fixes: rust-lang#125578 r? `@onur-ozkan`
Make mtime of reproducible tarballs dependent on git commit Since rust-lang#123246, our tarballs should be fully reproducible. That means that the mtime of all files and directories in the tarballs is set to the date of the first Rust commit (from 2006). However, this is causing some mtime invalidation issues (rust-lang#125578 (comment)). Ideally, we would like to keep the mtime reproducible, but still update it with new versions of Rust. That's what this PR does. It modifies the tarball installer bootstrap invocation so that if the current rustc directory is managed by git, we will set the UTC timestamp of the latest commit as the mtime for all files in the archive. This means that the archive should be still fully reproducible from a given commit SHA, but it will also be changed with new beta bumps and `download-rustc` versions. Note that only files are set to this mtime, directories are still set to the year 2006, because the `tar` library used by `rust-installer` doesn't allow us to selectively override mtime for directories (or at least I haven't found it). We could work around that by doing all the mtime modifications in bootstrap, but that would require more changes. I think/hope that just modifying the file mtimes should be enough. It should at least fix cargo `rustc` mtime invalidation. Fixes: rust-lang#125578 r? `@onur-ozkan` try-job: x86_64-gnu-distcheck
…zkan Make mtime of reproducible tarballs dependent on git commit Since rust-lang#123246, our tarballs should be fully reproducible. That means that the mtime of all files and directories in the tarballs is set to the date of the first Rust commit (from 2006). However, this is causing some mtime invalidation issues (rust-lang#125578 (comment)). Ideally, we would like to keep the mtime reproducible, but still update it with new versions of Rust. That's what this PR does. It modifies the tarball installer bootstrap invocation so that if the current rustc directory is managed by git, we will set the UTC timestamp of the latest commit as the mtime for all files in the archive. This means that the archive should be still fully reproducible from a given commit SHA, but it will also be changed with new beta bumps and `download-rustc` versions. Note that only files are set to this mtime, directories are still set to the year 2006, because the `tar` library used by `rust-installer` doesn't allow us to selectively override mtime for directories (or at least I haven't found it). We could work around that by doing all the mtime modifications in bootstrap, but that would require more changes. I think/hope that just modifying the file mtimes should be enough. It should at least fix cargo `rustc` mtime invalidation. Fixes: rust-lang#125578 r? `@onur-ozkan` try-job: x86_64-gnu-distcheck
…-ozkan Make mtime of reproducible tarballs dependent on git commit Since rust-lang#123246, our tarballs should be fully reproducible. That means that the mtime of all files and directories in the tarballs is set to the date of the first Rust commit (from 2006). However, this is causing some mtime invalidation issues (rust-lang#125578 (comment)). Ideally, we would like to keep the mtime reproducible, but still update it with new versions of Rust. That's what this PR does. It modifies the tarball installer bootstrap invocation so that if the current rustc directory is managed by git, we will set the UTC timestamp of the latest commit as the mtime for all files in the archive. This means that the archive should be still fully reproducible from a given commit SHA, but it will also be changed with new beta bumps and `download-rustc` versions. Note that only files are set to this mtime, directories are still set to the year 2006, because the `tar` library used by `rust-installer` doesn't allow us to selectively override mtime for directories (or at least I haven't found it). We could work around that by doing all the mtime modifications in bootstrap, but that would require more changes. I think/hope that just modifying the file mtimes should be enough. It should at least fix cargo `rustc` mtime invalidation. Fixes: rust-lang#125578 r? `@onur-ozkan` try-job: x86_64-gnu-distcheck
Rollup merge of rust-lang#127050 - Kobzol:reproducibility-git, r=onur-ozkan Make mtime of reproducible tarballs dependent on git commit Since rust-lang#123246, our tarballs should be fully reproducible. That means that the mtime of all files and directories in the tarballs is set to the date of the first Rust commit (from 2006). However, this is causing some mtime invalidation issues (rust-lang#125578 (comment)). Ideally, we would like to keep the mtime reproducible, but still update it with new versions of Rust. That's what this PR does. It modifies the tarball installer bootstrap invocation so that if the current rustc directory is managed by git, we will set the UTC timestamp of the latest commit as the mtime for all files in the archive. This means that the archive should be still fully reproducible from a given commit SHA, but it will also be changed with new beta bumps and `download-rustc` versions. Note that only files are set to this mtime, directories are still set to the year 2006, because the `tar` library used by `rust-installer` doesn't allow us to selectively override mtime for directories (or at least I haven't found it). We could work around that by doing all the mtime modifications in bootstrap, but that would require more changes. I think/hope that just modifying the file mtimes should be enough. It should at least fix cargo `rustc` mtime invalidation. Fixes: rust-lang#125578 r? ``@onur-ozkan`` try-job: x86_64-gnu-distcheck
|
Seems like this isn't fixed yet (see #129924 (comment)). |
Done by undoing temporary fix for rust-lang/rust#125578. Not yet fully verified, but this removal was there before and got lost when comments were added between versions.
I did a
git pull, fixed the submodules, and now I am trying to run./x.py check. However, that fails:This must b a fairly recent regression, I've never had to manually clean bootstrap before.
Cc @onur-ozkan
The text was updated successfully, but these errors were encountered: