Add cross compilation for OSX ARM64 (Apple Silicon)

[leolara]

What type of PR is this?

Feature on compilation infrastructure

What does this PR do? Why is it needed?

Implements cross compilation from Linux AMD64 to OSX ARM64 (Apple Silicon)

In more detail it:

• Updates dependency to osxcross to current master head
• Updates MacOS Sdk to v12.3 (I think Xcode v1.3.1)
• Updates clang cross compiler to v12.0.0

Which issues(s) does this PR fix?

Fixes #9385 #7950

Other notes for review

• This branch is based on this PR branch Upgrade Grail Bazel toolchain to latest release v0.7.2 #10950 to be able to work on Ubuntu 22.04. We could merge that one first, or it would be merged as part of this.
• The code in tool/cross-toolchain/config is autogenerated using tools/cross-toolchain/regenerate.sh
• I have not included the SDK, please send me a DM if you want the one I have or you need help generating it yourself.
• Once the new SDK has been uploaded to the prysm servers, we should update the code here to download it from there
• Once the docker image has been uploaded to the prysm registry, we need to update in the code
• It is enabled that you can add commits to this PR for these things, or I can do them myself
• I have worked on the docker-sandbox mode, but I guess this means it would work in local and with RBE. I do not know how to test for RBE, I don't know if somewhere it is explained their setup.
• I have run the script below to test that it builds in the 5 platforms, and tested that the binary for OSX ARM64 actually runs. The resulting binaries and logs are here: https://www.leolara.me/cross-artifacts.tar.gz

echo STARTING CROSS COMPILATION TEST

# linux_amd64
echo starting linux_amd64
rm -fR /home/leo/.cache/bazel bazel-* && bazel clean --expunge
mkdir -p cross-artifacts/linux_amd64
bazel build //cmd/validator //cmd/beacon-chain --config=linux_amd64_docker > cross-artifacts/linux_amd64/log 2>&1
cp bazel-bin/cmd/beacon-chain/beacon-chain_/beacon-chain cross-artifacts/linux_amd64/
cp bazel-bin/cmd/validator/validator_/validator cross-artifacts/linux_amd64/

# osx_amd64
echo starting osx_amd64
rm -fR /home/leo/.cache/bazel bazel-* && bazel clean --expunge
mkdir -p cross-artifacts/osx_amd64
bazel build //cmd/validator //cmd/beacon-chain --config=osx_amd64_docker >  cross-artifacts/osx_amd64/log 2>&1
cp bazel-bin/cmd/beacon-chain/beacon-chain_/beacon-chain cross-artifacts/osx_amd64/
cp bazel-bin/cmd/validator/validator_/validator cross-artifacts/osx_amd64/

# osx_arm64
echo starting osx_arm64
rm -fR /home/leo/.cache/bazel bazel-* && bazel clean --expunge
mkdir -p cross-artifacts/osx_arm64
bazel build //cmd/validator //cmd/beacon-chain --config=osx_arm64_docker > cross-artifacts/osx_arm64/log 2>&1
cp bazel-bin/cmd/beacon-chain/beacon-chain_/beacon-chain cross-artifacts/osx_arm64/
cp bazel-bin/cmd/validator/validator_/validator cross-artifacts/osx_arm64/

# linux_arm64
echo starting linux_arm64
rm -fR /home/leo/.cache/bazel bazel-* && bazel clean --expunge
mkdir -p cross-artifacts/linux_arm64
bazel build //cmd/validator //cmd/beacon-chain --config=linux_arm64_docker > cross-artifacts/linux_arm64/log 2>&1
cp bazel-bin/cmd/beacon-chain/beacon-chain_/beacon-chain cross-artifacts/linux_arm64/
cp bazel-bin/cmd/validator/validator_/validator cross-artifacts/linux_arm64/

# windows_amd64
echo starting windows_amd64
rm -fR /home/leo/.cache/bazel bazel-* && bazel clean --expunge
mkdir -p cross-artifacts/windows_amd64
bazel build //cmd/validator //cmd/beacon-chain --config=windows_amd64_docker > cross-artifacts/windows_amd64/log 2>&1
cp bazel-bin/cmd/beacon-chain/beacon-chain_/beacon-chain.exe cross-artifacts/windows_amd64/
cp bazel-bin/cmd/validator/validator_/validator.exe cross-artifacts/windows_amd64/

echo FINISHED ALL

TODO (could be done by core team or me, as you wish):

• Upload new MacOS SDK to prysmatic servers
• Upload new docker image to prysmatic registry
• Update code to download the MacOS SDK from prysmatic servers
• Update code to use the image from prysmatic registry

[prestonvanloon]

Hi, thanks so much for getting this to work! I have tried it locally and my eyes were pleased to see a completed osx arm64 binary.

Could you please provide instructions on how the SDK was generated? I can review that and then we can proceed with your nice list of action items. Thanks again, this is huge!

[leolara]

Hi, thanks so much for getting this to work! I have tried it locally and my eyes were pleased to see a completed osx arm64 binary.

Could you please provide instructions on how the SDK was generated? I can review that and then we can proceed with your nice list of action items. Thanks again, this is huge!

@prestonvanloon

These are more or less the steps I took to create the SDK on a AMD64 Ubuntu 22.04 system:

1. Go here https://developer.apple.com/download/all/?q=xcode
2. Log in with an Apple credential
3. Then it should show a list of downloables titled as "More downloads", find an Xcode package to download, I think I downloaded 13.3.1 but I am not sure, I think any 13.3.x should work. This should download a .xip file
4. Clone https://github.com/tpoechtrager/osxcross
5. from the root of osxcross repo run ./tools/gen_sdk_package_pbzx.sh xcode.xip where xcode.xip is the path to the xip file you just downloaded.
6. This will take a while depending on your machine.

This will produce a MacOSX12.3.sdk.tar.xz (if you used xcode 13.3 series, the SDK version should be 12.3).

Then we need to upload that somewhere so the script can download it, or include it in the repo, whatever you think it is best.

[prestonvanloon]

Working on review of this now.

[prestonvanloon]

Absolutely amazing work! Thanks so much for this change and your patience with code review.

These changes look great. The only remaining task is that I will need to build a new image for our CI runner as it is an extension of the rbe-worker image. Until then, we will see a few tasks failing in buildkite.

[rauljordan]

Seems like ci fails to build some images @prestonvanloon @leolara

[leolara]

Yes, @prestonvanloon mentioned that this would happen in previous comment. I think the docker image should be built and uploaded to your registry. Then the code changed to point to your registry