Tracking issue for supporting macOS on Apple Silicon (a.k.a arm64, M1, M2, aarch64)

[shepmaster]

On 2020-06-22, Apple announced that future computers will utilize Arm processors instead of x86_64 processors. This is the tracking issue for supporting that platform, commonly referred to as Apple Silicon.

⚠️ Don't rely on this thread to decide whether to purchase Apple Silicon hardware. ⚠️
The Rust project does not make any guarantee on the level of support the compiler has.

Current status summary (2022-07-05)

• There is no timeline for any level of support.

• aarch64-apple-darwin is a tier 2 target and is available for download via rustup. This compiler can be used to cross-compile from x86_64-apple-darwin as well as running natively on Arm.

• Compiler tests are not being run at this point, as CI has no way of executing them. This is a blocker for Tier 1 support.

• The x86_64 version of rustc works under Rosetta, producing x86_64 binaries that themselves run under Rosetta.

About tracking issues

Tracking issues are used to record the overall progress of implementation. They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.

A tracking issue is not meant for large scale discussion, questions, or bug reports about a feature. Instead, join the discussion and/or contribute in the #t-compiler/arm Zulip stream or open a new issue for bug reports. You are encouraged to comment here to link to related issues.

This tracking issue will be closely moderated in an attempt to maintain a high signal-to-noise ratio.

Steps

This is not yet intended to be an exhaustive or even accurate list of steps.

• Find out if there is anyone inside the compiler team with the time to drive the effort forward. This will allow us to efficiently allocate the Developer Transition Kits (DTK), if we end up receiving any. There are no members of the compiler team actively working on this
• Reach out to our contacts at Apple to see if DTKs are available, and to establish a line of communication if there are problems down the line. Please coordinate with the leads of the compiler and infra teams on Zulip on this.
• Achieve Tier 3 support:
  • Wait for proper support to be upstreamed into LLVM, and update the Rust compiler to pull the latest version of LLVM.
    • Our current LLVM supports targeting the appropriate hardware well enough.
  • Update the libc crate's conditional compilation to account for the new target.
    • Limit macOS $INODE64 symbol names to x86 and x86_64 libc#1817 (in libc 0.2.73)
  • Update the cc crate to account for the new target.
    • Always specify architecture for x86_64 / aarch64 Darwin cc-rs#527 (in cc 1.0.58)
  • Submit an initial target specification for aarch64-apple-darwin.
    • Add the aarch64-apple-darwin target  #74541
  • Get the target specification into the bootstrap compiler.
    • Add the aarch64-apple-darwin target  #74541
    • [beta] backports #75261
• Achieve Tier 2 support:
  • Get Xcode 12 on our CI
  • Update the openssl-src crate to account for the new target.
    • Add upstream patch to allow building on aarch64-apple-darwin alexcrichton/openssl-src-rs#74
  • Setup a cross-compiling builder to reach Tier 2
    • Add to CI (Set up CI for aarch64-apple-darwin #75991)
    • Build Cargo (Enable building Cargo for aarch64-apple-darwin #77239)
    • Distribute binaries (Include aarch64-apple-darwin in the dist manifests #77895)
• Achieve Tier 1 support:
  • Entire test suite passes
    • Variadic FFI test fails on aarch64-apple-darwin #78092
    • LLVM assertion failure after activating LTO on aarch64-apple-darwin #78881
  • Increase the quality and reliability of the target
  • Hook up target hardware to our CI, and start running the test suite as part of our standard QA process.
  • Propose the promotion to Tier 1.
• Miscellaneous
  • Rustup
    • Add support to rustup (Build for aarch64-apple-darwin rustup#2521)
    • Release it (Provide full Apple Silicon (arm64/AArch64-Apple-Darwin) support in rustup rustup#2413)

Resolved Questions

• The infra team does not want to maintain our own hardware / VMs. This means solutions like MacStadium, AWS, or "putting our own hardware under someone's desk" are non-starters.

Unresolved Questions

This is not yet intended to be an exhaustive or even accurate list of questions.

• When will our CI providers have access to native hardware?
  • Support for VMs on Apple M1 actions/runner-images#2187
  • GitHub Actions: Apple Silicon (M1) powered macOS runners (Public Beta) github/roadmap#528
• Should we use an alternative / additional CI provider to GitHub?
  • Cirrus CI
• What is an accurate target name for the new platform?
  • What is the accurate name of the chip ("silicon", "aarch64", "arm64", etc.)?
• How can we leave room to support Apple's arm64e?

Implementation history

[Absolucy]

How can we leave room to support Apple's arm64e?

I am currently adding PAC/arm64e to a fork using Apple's LLVM. Hopefully some of my work can be reused whenever PAC/arm64e support is upstreamed into LLVM.

[glandium]

Not sure if I should open a separate issue, but running the x86-64 rustc on a DTK (via Rosetta 2) yields <jemalloc>: Unsupported system page size messages (but things still work), so it's likely that the arm64 port would need some adjustment in that area. (Not sure if the x86-64 rustc should be adjusted to to work without those messages under Rosetta 2)

(The issue being that the page size is 16KB instead of 4KB)

[glandium]

For the record, I've built the rust compiler on the DTK with #75500, a tweak to stage0.txt to pick a recent nightly, and a config.toml that sets the following:

build = "x86_64-apple-darwin"
host = ["aarch64-apple-darwin"]
target = ["aarch64-apple-darwin"]

and was subsequently able to build all the rust code in Firefox but I had to make changes to three crates that were passing the wrong --target to clang because the rust target differs from the clang target (rust uses aarch64-apple-darwin, clang uses arm64-apple-darwin). This is very unfortunate, and it's probably going to cause problems in other crates.

[pnkfelix]

I'm throwing a link to #74820 on here because I suspect we'll want to better understand that bug before we can stabilize a Tier 1 ARM target.

[glandium]

Get Xcode 12 on our CI, and setup a cross-compiling builder to reach Tier 2.

It's already there, both on github actions and azure pipelines, per https://github.com/actions/virtual-environments/blob/main/images/macos/macos-10.15-Readme.md and https://github.com/microsoft/azure-pipelines-image-generation/blob/master/images/macos/macos-10.15-Readme.md (which look like to be the same document, btw)

[shepmaster]

Status update

Progress has continued at a measured pace. You can see a summary of the work to date in the "Current status summary" section in the top post. The most recent update is that aarch64-apple-darwin is now a Tier 2 platform. It is important to note that the compiler tests are not being run at this point, as CI has no way of executing them.

If you'd like to try using the new compiler, I've created some basic instructions on how to configure your project to cross-compile to aarch64-apple-darwin. If you have access to an Apple DTK, there are also instructions on how to download and install the native toolchain.

---

As a reminder:

A tracking issue is not meant for large scale discussion, questions, or bug reports about a feature. Instead, join the discussion and/or contribute in the #t-compiler/arm Zulip stream or open a new issue for bug reports. You are encouraged to comment here to link to related issues.

[lunabunn]

The Rust Infrastructure Team has regular calls with GitHub, and M1 runners are a recurrent topic of those calls. People outside the Infrastructure Team setting up backchannels does not help getting M1 runners faster, it just makes the situation confusing and hurts the relationship Rust has built with GitHub.

Unless I am misunderstanding, there has not been a single message from you or any other member of the infrastructure team (except for, of course, shepmaster) in this entire issue thread. While I fully understand that internal discussion happens on Zulip and that fractured efforts are oftentimes worse than no effort, it is difficult to comprehend your mention of "backchannels" when said "backchannel" was communicated here and the "recurrent ... relationship Rust has built with GitHub" has not. The most status update even explicitly points out the lack of CI infrastructure as a blocker.

Please remember that we (the community -- though I myself am not a direct contributor to rust-lang/rust) use and contribute to Rust and its ecosystem with good intentions and depend on active communication from the Rust teams to coordinate our efforts ❤️

[darthdeus]

@aviramha please refrain from contacting GitHub (or any other company) on behalf of the Rust project.

The Rust Infrastructure Team has regular calls with GitHub, and M1 runners are a recurrent topic of those calls. People outside the Infrastructure Team setting up backchannels does not help getting M1 runners faster, it just makes the situation confusing and hurts the relationship Rust has built with GitHub.

Please stop the conversation you're having and redirect it to [email protected] (if that's via email) or to @pietroalbini on Twitter (if it's on Twitter, so that I can route it to the most appropriate place). Thank you.

Pietro. Rust Infrastructure Team Lead

I'm not sure what you're hoping to achieve with comments like this other than alienate possible contributors. People in the community are having problems, those people are trying to resolve problems on their own because there is no other apparent progress. If the infrastructure team has been working on resolving the problem behind closed doors and nobody knows that it's being worked on, maybe it shouldn't be surprising that the community is trying to resolve the problem on their own.

What actually hurts Rust the language is the constant drama from lacking communication on the leadership side. Comments like this actively discourage any active contributors and add to the already pretty big fire around the Rust project.

Many important people have already left Rust on not so good terms based on how they've been treated by the leadership. Comments like this are only going to make more people leave.

[infiniteregrets]

@aviramha please refrain from contacting GitHub (or any other company) on behalf of the Rust project.

I don't understand this negative enforcement of efforts made by the community, the intentions here, clearly were to help

[tarcieri]

Can we please keep discussion on this issue focused on technical topics? You're spamming my inbox with an irrelevant discussion.

[pietroalbini]

Sorry about this. I didn't contact in name of the organization nor the foundation. I just asked if someone can help as the latest replies asked about having someone sponsor it. I'll send him your handle.

On a personal note I think the tone of your message is quite discouraging while I understand my attempt might have been annoying it was out of desire to help and I didn't have any bad intentions.

First off, I'm sorry for the tone in my message. I didn't have much energy left to handle this after dealing with the keynote situation for so much time, and quickly wrote a message to limit potential damage. I should've wrote it in a way that didn't imply bad intentions on your end (as it was clear you were just trying to help).

Thankfully no real problem seems to have happened from this (after checking with GitHub folks) 🙂

In general, before reaching out it'd be good to check with the maintainers of a project to see if they're already in contact with that entity. It didn't happen in this case, but sometimes when someone higher up notices a problem and they escalate it internally, causing trouble for the folks the project has been in contact with. Checking beforehand helps prevent that from happening. Again, thankfully it didn't happen with GitHub, but that was the risk I was trying to mitigate.

While I fully understand that internal discussion happens on Zulip and that fractured efforts are oftentimes worse than no effort, it is difficult to comprehend your mention of "backchannels" when said "backchannel" was communicated here and the "recurrent ... relationship Rust has built with GitHub" has not. The most status update even explicitly points out the lack of CI infrastructure as a blocker.

If the infrastructure team has been working on resolving the problem behind closed doors and nobody knows that it's being worked on, maybe it shouldn't be surprising that the community is trying to resolve the problem on their own.

We should've posted one update in this issue that we were in communication with GitHub, and that's our fault. There isn't much more we're actively doing on this right now or that we can share though, as GitHub is sharing with us confidential updates on their progress.

Regarding Rust's relationship with GitHub in general, we've been having regular topics in the t-infra stream on Zulip to ask the project members issues to raise to GitHub, so I wouldn't say it's a closed backchannel (even though we cannot share everything publicly due to confidentiality). It's not much relevant outside of the project except for this issue, where as I already said we should've communicated better.

Again, sorry @aviramha for the tone of my message.

Can we please keep discussion on this issue focused on technical topics? You're spamming my inbox with an irrelevant discussion.

If you want to followup please do so on the t-infra stream on Zulip, so that we can avoid further notifications.

[mlindner]

@pietroalbini Seeing as you appear to be in the know, can you please explain the current status? The top comment hasn't been updated in about 9 months. This is a relatively common topic of conversation so if you know something, please mention it. "We are having conversations" doesn't really say anything. (In general this whole Zulip this Zulip that is a huge negative of the Rust project. Stick to GitHub.)

Edit: Side note, reviewing the top comment and there's an interesting bit of irony that the infrastructure team doesn't want to maintain actual physical infrastructure (the meaning of the word infrastructure implies something physical). Maybe a decision that needs to be re-evaluated rather than expecting GitHub to provide every piece of infrastructure. The Linux project seems to manage it somehow.

[shepmaster]

The top comment hasn't been updated in about 9 months

That is because there is no update to the status.

I have been using Apple hardware since I was 4 years old. I worked with people with access to the developer kits to get Rust compiling on / for Apple Silicon as soon as possible. I'd believe I want this issue closed more than the vast majority of people subscribed.

As soon as something worth posting is available and I'm awake / available / near a computer, I will post it!

"We are having conversations" doesn't really say anything

I am not going to post a status update every week / two weeks / month that says "no progress". That will drive people batty and provides no information. GitHub has been a gracious sponsor and the Rust project uses a lot of GitHub functionality; we are lucky to have a few points of contact that we communicate with on an ongoing basis (think email / virtual meetings — I actually choose to not attend those meetings so I'm not 100% sure).

Anything that GitHub chooses to say will be on their public issues, which is why we've linked them above. You are highly encouraged to subscribe there and you will know the same things we do as soon as we do.

In general this whole Zulip this Zulip that is a huge negative of the Rust project. Stick to GitHub.

The Rust Zulip is open to read. Posting does require signing up, but you can sign up using your existing GitHub account. Sometimes communication is longer form and sometimes it's shorter form; that's just how we humans work. Zulip represents a different speed of communication than GitHub.

Also, sometimes communication needs to be private. This is especially true for a team like the infrastructure team, which has to deal with accounts or passwords or signing keys, etc. Just as you don't post all of your usernames and passwords to a publicly available GitHub repository, neither do we. I get that this can be frustrating — I wish I could know every sordid detail of GitHub's M1 implementation, but I don't get to and that's just something I have to deal with.

the infrastructure team doesn't want to maintain actual physical infrastructure

The team is small and has a finite amount of time and experience so we have to prioritize what we work on. Creating a production-hardened macOS CI system that's open to arbitrary attackers from the internet is not trivial and is not cheap.

the meaning of the word infrastructure implies something physical

One thing that I have had to deal with is that words are created or change and take on new meanings (no cap, fam). Other than my own machines, I think I last helped rack a physical server about twenty years ago. I can't imagine any of our volunteers hopping into their car and driving down to the colo to swap out a hard drive after their pager woke them up at 3 in the morning so that every Rust build in the world can continue.

expecting GitHub to provide every piece of infrastructure

I don't disagree: having our eggs in multiple baskets would be nice. However, this comes back to the time tradeoff. The Rust build is nontrivial and there's no easy button to press to make it distributed across multiple CI providers. We've actually had that in the past and breathed a sigh of relief when we could consolidate back to one provider.

The Linux project seems to manage it somehow

Certainly, this isn't something impossible that breaks the laws of physics, it's just hard. I have a feeling that Linux is a bigger project and better resourced.

---

If you are thinking to yourself "I've got a good amount of time to contribute and have the skills to maintain / improve Rust's infrastructure", then by all means come on over to Zulip and talk with us.

If you are a company that provides Apple Silicon macOS CI and want to donate your services, talk to us and/or the Rust Foundation and we'll see if we can make anything happen.

Ultimately, please remember that the infra team is composed of humans with lives and thoughts and feelings. We aren't hiding things from you, we aren't maliciously degrading Rust CI, etc. I'm on vacation this week and instead of spending time with my daughter this morning, I'm replying to this issue in the hopes of reducing tensions so that there's not a bigger fire later on. I hope that I don't regret this morning's time choices.

[mlindner]

Thank you for the long and detailed response. Apologies from taking you away from your vacation. Please have fun and have a nice day.

[shepmaster]

GitHub recently announced the public beta availability of Apple Silicon builders for GitHub Actions. We are as excited as you are about this news, and we plan on trying them out as soon as possible! After we have had a chance to do so, we'll be sure to report back here. Thanks for hanging in there for as long as you have, and we hope to have more good news in the near future.

[rami3l]

Small update: Recently we have been running the full Rustup test suite on aarch64-apple-darwin with the new macos-14 runner and so far it's doing great: rust-lang/rustup#3681 🎉

[shepmaster]

I have opened RFC 3671 to promote aarch64-apple-darwin to Tier 1.

[evieluvsrainbows]

PR #128592 promotes aarch64-apple-darwin to Tier 1, as per the newly merged RFC.

[evieluvsrainbows]

As of today, ARM64 macOS (Darwin) is an officially recognized tier 1 target. I am proposing that this issue be closed now as the main goal of having arm64 macOS be an officially supported Tier 1 target has been reached.

[ehuss]

Thanks @evelynharthbrooke! I'll go ahead and close as completed by #128592.