Fail to build cpp-tutorial stage1 with Bazel 7.0.0 #20533

sjwo · 2023-12-13T19:54:55Z

Description of the bug:

Bazel 7.0.0 fails to build the stage1 portion of the bazelbuild/examples/cpp-tutorial. It appears to fail to resolve a path it constructs.

Reverting to Bazel 6.4.0, or passing the --noincompatible_sandbox_hermetic_tmp build option, are both successful workarounds.

Which category does this issue belong to?

No response

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Ubuntu 20.04.
Download bazelisk-linux-amd64 from https://github.com/bazelbuild/bazelisk/releases/tag/v1.19.0, add it to your PATH, and make it executable.
Clone https://github.com/bazelbuild/examples, and change into the examples/cpp-tutorial/stage1 directory.
Execute bazelisk-linux-amd64 build //main:hello-world.
As part of the build failure output, receive the following error message:

external/gcc_toolchain_x86_64/bin/gcc: line 45: /tmp/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-gcc: No such file or directory

Which operating system are you running Bazel on?

Ubuntu 20.04

What is the output of `bazel info release`?

release 7.0.0

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

No response

What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

https://github.com/bazelbuild/examples
master
fatal: ambiguous argument 'master': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
9c4e6c5de1a053efa956c0a7a8f54d302d24e0b1

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

Possibly.

Have you found anything relevant by searching the web?

My post to the Bazel getstarted Slack includes responses from other users suggesting the workarounds mentioned above: https://bazelbuild.slack.com/archives/CD42R4HJL/p1702485704389129
This issue over at the examples repo appears to be having a similar and possibly the same issue: cpp-tutorial can't find external linux toolchain examples#383

Any other information, logs, or outputs that you want to share?

Here is the output of git rev-parse main:

9c4e6c5de1a053efa956c0a7a8f54d302d24e0b1

The text was updated successfully, but these errors were encountered:

fmeum · 2023-12-19T12:29:38Z

I verified that this issue is still present with #20603.

I dug further and think that this logic just doesn't work with the kind of relocation performed by --incompatible_sandbox_hermetic_tmp: https://github.com/aspect-build/gcc-toolchain/blob/e2a34698bd20c7a2f771586ddfcf504330b77a97/toolchain/wrapper.sh.tpl#L20-L25

I don't think that this is a bug in Bazel, it's only an incompatible change to some aspects of the path layout. CC @f0rmiga

f0rmiga · 2023-12-19T17:22:25Z

Thanks @fmeum. I'll take a look at that.

dubrovin-sudo · 2024-02-16T13:10:06Z

This bug still exist
Ubuntu: 22.4
Bazel: 7.0.2
GCC: 11.4.0

jeangud · 2024-02-22T15:09:35Z

FYI looks like the --sandbox_add_mount_pair=/tmp build option is also a workaround.

From: #19915

pawsaw · 2024-03-14T11:49:32Z

Ubuntu 22.4
Bazel: 7.1.0

The workaround as suggested by @jeangud works for me --sandbox_add_mount_pair=/tmp

Still open. Why is this closed? @fmeum

fmeum · 2024-03-14T12:14:05Z

@pawsaw I closed it as not planned since it's more of a bug in the gcc toolchain that isn't part of Bazel. But it's still referenced by Bazel examples, so I now agree that it's better to reopen the bug.

@iancha1992 Could you reopen this issue? I don't have permission to do so.

comius · 2024-05-03T13:53:14Z

cc @lberki for incompatible_sandbox_hermetic_tmp

f0rmiga · 2024-05-03T20:31:41Z

I have the bandwidth to look into it today on my end. As for Bazel itself, I don't know if there's an action to take.

The bind mounting scheme used with the Linux sandbox' hermetic `/tmp` feature is modified to preserve all paths as they are outside the sandbox, which removes the need to rewrite paths when staging inputs into and, crucially, moving outputs out of the sandbox. Source roots and output base paths under `/tmp` are now treated just like any user-specified bind mount under `/tmp`: They are mounted under the hermetic tmp directory with their path relativized against `/tmp` before the hermetic tmp directory is mounted as `/tmp` as the final step. There is one caveat compared to user-specified mounts: Source roots, which may themselves not lie under `/tmp`, can be symlinks to directories under `/tmp` (e.g., when they arise from a `local_repository`). To handle this situation in the common case, all parent directories of package path entries (up to direct children of `/tmp`) are mounted into the sandbox. If users use `local_repository`s with fixed target paths under `/tmp`, they will need to specify `--sandbox_add_mount_pair`. Overlayfs has been considered as an alternative to this approach, but ultimately doesn't seem to work for this use case since its `lowerpath`, which would be `/tmp`, is not allowed to have child mounts from a different user namespace (see https://unix.stackexchange.com/questions/776030/mounting-overlayfs-in-a-user-namespace-with-child-mounts). However, this is exactly the situation created by a Bazel-in-Bazel test and can also arise if the user has existing mounts under `/tmp` when using Bazel (e.g. the JetBrains toolbox on Linux uses such mounts). This replaces and mostly reverts the following commits, but keeps their tests: * bazelbuild@bf6ebe9 * bazelbuild@fb6658c * bazelbuild@bc1d9d3 * bazelbuild@1829883 * bazelbuild@70691f2 * bazelbuild@a556969 * bazelbuild@8e32f44 (had its test lost in an incorrect merge conflict resolution, this PR adds it back) Fixes bazelbuild#20533 Work towards bazelbuild#20753 Fixes bazelbuild#21215 Fixes bazelbuild#22117 Fixes bazelbuild#22226 Fixes bazelbuild#22290 RELNOTES: Paths in the Linux sandbox are now again identical to those outside the sandbox, even with `--incompatible_sandbox_hermetic_tmp`. Closes bazelbuild#22001. PiperOrigin-RevId: 634381503 Change-Id: I9f7f3948c705be120c55c9b0c51204e5bea45f61

The bind mounting scheme used with the Linux sandbox' hermetic `/tmp` feature is modified to preserve all paths as they are outside the sandbox, which removes the need to rewrite paths when staging inputs into and, crucially, moving outputs out of the sandbox. Source roots and output base paths under `/tmp` are now treated just like any user-specified bind mount under `/tmp`: They are mounted under the hermetic tmp directory with their path relativized against `/tmp` before the hermetic tmp directory is mounted as `/tmp` as the final step. There is one caveat compared to user-specified mounts: Source roots, which may themselves not lie under `/tmp`, can be symlinks to directories under `/tmp` (e.g., when they arise from a `local_repository`). To handle this situation in the common case, all parent directories of package path entries (up to direct children of `/tmp`) are mounted into the sandbox. If users use `local_repository`s with fixed target paths under `/tmp`, they will need to specify `--sandbox_add_mount_pair`. Overlayfs has been considered as an alternative to this approach, but ultimately doesn't seem to work for this use case since its `lowerpath`, which would be `/tmp`, is not allowed to have child mounts from a different user namespace (see https://unix.stackexchange.com/questions/776030/mounting-overlayfs-in-a-user-namespace-with-child-mounts). However, this is exactly the situation created by a Bazel-in-Bazel test and can also arise if the user has existing mounts under `/tmp` when using Bazel (e.g. the JetBrains toolbox on Linux uses such mounts). This replaces and mostly reverts the following commits, but keeps their tests: * bf6ebe9 * fb6658c * bc1d9d3 * 1829883 * 70691f2 * a556969 * 8e32f44 (had its test lost in an incorrect merge conflict resolution, this PR adds it back) Fixes #20533 Work towards #20753 Fixes #21215 Fixes #22117 Fixes #22226 Fixes #22290 RELNOTES: Paths in the Linux sandbox are now again identical to those outside the sandbox, even with `--incompatible_sandbox_hermetic_tmp`. Closes #22001. PiperOrigin-RevId: 634381503 Change-Id: I9f7f3948c705be120c55c9b0c51204e5bea45f61 Fixes #22291

sjwo added type: bug untriaged labels Dec 13, 2023

sjwo assigned iancha1992, rakshitasingh05 and sgowroji Dec 13, 2023

iancha1992 unassigned sgowroji, iancha1992 and rakshitasingh05 Dec 14, 2023

iancha1992 added the team-Rules-CPP Issues for C++ rules label Dec 14, 2023

fmeum mentioned this issue Dec 19, 2023

[bazel.build] Problem with /start/cpp #20607

Closed

fmeum closed this as not planned Won't fix, can't repro, duplicate, stale Dec 19, 2023

iancha1992 reopened this Mar 14, 2024

prasad-shirodkar mentioned this issue Apr 15, 2024

cpp-tutorial/stage1 bazel build looks for compiler in wrong /tmp path bazelbuild/examples#394

Open

fmeum mentioned this issue May 8, 2024

Preserve paths under hermetic /tmp in the sandbox #22001

Closed

comius added P3 We're not considering working on this, but happy to review a PR. (No assignee) and removed untriaged labels May 15, 2024

copybara-service bot closed this as completed in 3fddc7f May 16, 2024

fmeum mentioned this issue May 16, 2024

[7.2.0] Preserve paths under hermetic /tmp in the sandbox #22407

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fail to build cpp-tutorial stage1 with Bazel 7.0.0 #20533

Fail to build cpp-tutorial stage1 with Bazel 7.0.0 #20533

sjwo commented Dec 13, 2023 •

edited

Loading

fmeum commented Dec 19, 2023

f0rmiga commented Dec 19, 2023

dubrovin-sudo commented Feb 16, 2024

jeangud commented Feb 22, 2024

pawsaw commented Mar 14, 2024 •

edited

Loading

fmeum commented Mar 14, 2024

comius commented May 3, 2024

f0rmiga commented May 3, 2024

Fail to build cpp-tutorial stage1 with Bazel 7.0.0 #20533

Fail to build cpp-tutorial stage1 with Bazel 7.0.0 #20533

Comments

sjwo commented Dec 13, 2023 • edited Loading

Description of the bug:

Which category does this issue belong to?

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Which operating system are you running Bazel on?

What is the output of bazel info release?

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

Have you found anything relevant by searching the web?

Any other information, logs, or outputs that you want to share?

fmeum commented Dec 19, 2023

f0rmiga commented Dec 19, 2023

dubrovin-sudo commented Feb 16, 2024

jeangud commented Feb 22, 2024

pawsaw commented Mar 14, 2024 • edited Loading

fmeum commented Mar 14, 2024

comius commented May 3, 2024

f0rmiga commented May 3, 2024

sjwo commented Dec 13, 2023 •

edited

Loading

What is the output of `bazel info release`?

If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

What's the output of `git remote get-url origin; git rev-parse master; git rev-parse HEAD` ?

pawsaw commented Mar 14, 2024 •

edited

Loading