-
Notifications
You must be signed in to change notification settings - Fork 740
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add ARM binaries to releases #802
Comments
The reason why we release only X86 binaries is that validators usually don't run nodes on anything else than Linux servers. When building a node is an annoyance (it takes a while) and devs just need to run a node, there is also the option to use the Docker image we provide. Could you provide details if your use case does not fall in the categories above ? |
Note that the linux x64 docker image should also run fine on m1 macs (tested with a current dev build of podman w/ qemu) |
I just came back from the Parachains Summit, and there were others asking for ARM builds, specifically to speed up development, since you need a polkadot binary to run a test relay chain |
AWS Graviton arm server is getting popular and it will be great if we can run polkadot nodes with it. |
+1 |
Any idea when the next release containing the arm64 binaries will come out? Will it target |
We are looking into it.
no. |
+1 "Our latest cost analysis and performance benchmarks show that customers can realize cost savings of up to 23% and performance gains of up to 36% by deploying the GitLab application and GitLab Runner on the Arm-based Graviton2 when compared to the x86 based M5/C5 EC2 instances." - https://about.gitlab.com/blog/2021/08/05/achieving-23-cost-savings-and-36-performance-gain-using-gitlab-and-gitlab-runner-on-arm-neoverse-based-aws-graviton2-processor/ |
+1 |
I think its being worked on here paritytech/polkadot#6108 @chevdor |
Indeed, I am on the topic atm. If someone wants to test, feel free to ping me, I can provide an unofficial yet built in production profile for |
@chevdor I am quite interested in testing out this arm64 image. Can you please let me know how get this image (is there a docker image available?) |
There is no docker image yet. I can provide a production build of an early RC of v0.9.38 for The version linked below is:
You can get the build here: https://github.com/chevdor/polkadot/releases/tag/v0.9.38-beta That being said, I would definitely recommend against considering the arm64 binary for a validator. If you find a provider with good specs and price, please share your findings (I checked GCP and AWS). Besides that, the build did work fine. I tested mostly wrap syncs with ParityDB and upgrading from v0.9.37 as well. |
Thanks for your work @chevdor. Do you plan to publish another build for M2/M2Pro? |
You can try to use my scripts to build an aarch64 binary that is optimized for your CPU. Of course they are not suitable for production builds. |
Hi everyone! Any updates or approximate ETA on this? Found out that it would be great to have an ability to launch zombienet kubernetes configuration locally having the Mac M1 with Docker Desktop based kubernets that runs ARM64 node. Currently I was able to make changes to Parity base-ci-linux and ci-linux Dockerfile to build them for ARM64 and succeded. Here are the links to images: base-ci-linux - required a single line of code to make it work cross-platform I can provide PRs if someone interested - just don't want to litter parity github repos. I don't recommend to use the in production for sure, but for local development might be helpful. |
There is some work probably interesing following in https://github.com/paritytech/release-engineering/issues/194 Atm, if this is just to make it convenient, you could make a build or used one of those I published. |
Could you share the code changes with me please? |
Sure, here are the links: |
* Benchmark with main spec * Update benchmark.md
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.56 to 0.1.57. - [Release notes](https://github.com/dtolnay/async-trait/releases) - [Commits](dtolnay/async-trait@0.1.56...0.1.57) --- updated-dependencies: - dependency-name: async-trait dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Add all members. * Change members to globs. * Remove runtime-common.
We really need this. 99% of our devs are probably using a Mac to develop on Polkadot-SDK. Developer experiences are worse because of lacking native binaries:
|
@shawntabrizi {
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
rust-overlay = {
url = "github:oxalica/rust-overlay";
inputs = {
nixpkgs.follows = "nixpkgs";
};
};
zombienet = {
url = "github:paritytech/zombienet/dfc0f2e02dbab2361c3f2f983fe6e0f32261cc73"; # last known working version, because of: https://github.com/paritytech/zombienet/issues/1858
inputs = {
nixpkgs.follows = "nixpkgs";
};
};
};
outputs = { self, nixpkgs, flake-utils, rust-overlay, zombienet }:
flake-utils.lib.eachDefaultSystem (system:
let
overlays = [ (import rust-overlay) zombienet.overlays.default ];
pkgs = import nixpkgs {
inherit system overlays;
};
rustToolchain = pkgs.pkgsBuildHost.rust-bin.fromRustupToolchainFile ./rust-toolchain.toml;
buildInputs = with pkgs; [
# Building Docker images and publishing to Azure Container Registry
azure-cli
# cargo-llvm-cov
cargo-expand
clang
pkg-config
rustToolchain
subxt
just
taplo
polkadot
mdbook
# Due to zombienet's flake.nix, needs to be prefixed with pkg.zombienet
pkgs.zombienet.default
]
++ (lib.optionals stdenv.isDarwin [
darwin.apple_sdk.frameworks.Security
darwin.apple_sdk.frameworks.CoreServices
darwin.apple_sdk.frameworks.SystemConfiguration
])
++ (lib.optionals stdenv.isLinux [
cargo-llvm-cov
]);
in
with pkgs;
{
devShells.default = mkShell {
inherit buildInputs;
LIBCLANG_PATH = "${llvmPackages.libclang.lib}/lib";
PROTOC = "${protobuf}/bin/protoc";
RUST_SRC_PATH = "${rustToolchain}/lib/rustlib/src/rust/library/";
ROCKSDB_LIB_DIR = "${rocksdb}/lib";
};
}
);
} ^ this flake allows you to run zombienet natively on MacOS with running command like: zombienet -p native spawn zombienet/local-testnet.toml |
Worth noting this is no2 in the developer wishlist :) #3900 |
what is the blocker? i dont get how this isn't just done. |
Would be great if we know if this is addressed by the next stable release or not. Given the 3 months release cadence, if the effort needed for it is minimal, I think it is reasonable to prioritize it in the next 2 weeks. |
I just had a chat with @EgorPopelyaev , the team has some capacity to take care of it next week. CI/CD ARM runner is in place, what is left is to add step to the binary build pipeline which is going to do an ARM build. @EgorPopelyaev would this fit in the timeline set by Kian? Let me know anything you need for this and I'll check with other teams who can assist. |
# Description This PR adds the required changes to release `polkadot`, `polkadot-parachain` and `polkadot-omni-node` binaries built on Apple Sillicon macos. ## Integration This addresses requests from the community for such binaries: #802, and they should be part of the Github release page. ## Review Notes Test on paritytech-stg solely focused on macos binaries: https://github.com/paritytech-stg/polkadot-sdk/actions/runs/11824692766/job/32946793308, except the steps related to `pgpkms` (which need AWS credentials, missing from paritytech-stg). The binary names don't have a `darwin-arm` identifier, and conflict with the existing x86_64-linux binaries. I haven't tested building everything on `paritytech-stg` because the x86_64-linux builds run on `unbutu-latest-m` which isn't enabled on `pairtytech-stg` (and I haven't asked CI team to enable one), so testing how to go around naming conflicts should be covered next. ### TODO - [x] Test the workflow start to end (especially the last bits related to uploading the binaries on S3 and ensuring the previous binaries and the new ones coexist harmoniously on S3/action artifacts storage without naming conflicts) @EgorPopelyaev - [x] Publish the arm binaries on the Github release page - to clarify what's needed @iulianbarbu . Current practice is to manually publish the binaries built via `release-build-rc.yml` workflow, taken from S3. Would be great to have the binaries there in the first place before working on automating this, but I would also do it in a follow up PR. ### Follow ups - [ ] unify the binaries building under `release-30_publish_release_draft.yml` maybe? - [ ] automate binary artifacts upload to S3 in `release-30_publish_release_draft.yml` --------- Signed-off-by: Iulian Barbu <[email protected]> Co-authored-by: EgorPopelyaev <[email protected]>
# Description This PR adds the required changes to release `polkadot`, `polkadot-parachain` and `polkadot-omni-node` binaries built on Apple Sillicon macos. ## Integration This addresses requests from the community for such binaries: #802, and they should be part of the Github release page. ## Review Notes Test on paritytech-stg solely focused on macos binaries: https://github.com/paritytech-stg/polkadot-sdk/actions/runs/11824692766/job/32946793308, except the steps related to `pgpkms` (which need AWS credentials, missing from paritytech-stg). The binary names don't have a `darwin-arm` identifier, and conflict with the existing x86_64-linux binaries. I haven't tested building everything on `paritytech-stg` because the x86_64-linux builds run on `unbutu-latest-m` which isn't enabled on `pairtytech-stg` (and I haven't asked CI team to enable one), so testing how to go around naming conflicts should be covered next. ### TODO - [x] Test the workflow start to end (especially the last bits related to uploading the binaries on S3 and ensuring the previous binaries and the new ones coexist harmoniously on S3/action artifacts storage without naming conflicts) @EgorPopelyaev - [x] Publish the arm binaries on the Github release page - to clarify what's needed @iulianbarbu . Current practice is to manually publish the binaries built via `release-build-rc.yml` workflow, taken from S3. Would be great to have the binaries there in the first place before working on automating this, but I would also do it in a follow up PR. ### Follow ups - [ ] unify the binaries building under `release-30_publish_release_draft.yml` maybe? - [ ] automate binary artifacts upload to S3 in `release-30_publish_release_draft.yml` --------- Signed-off-by: Iulian Barbu <[email protected]> Co-authored-by: EgorPopelyaev <[email protected]>
This PR adds the required changes to release `polkadot`, `polkadot-parachain` and `polkadot-omni-node` binaries built on Apple Sillicon macos. This addresses requests from the community for such binaries: #802, and they should be part of the Github release page. Test on paritytech-stg solely focused on macos binaries: https://github.com/paritytech-stg/polkadot-sdk/actions/runs/11824692766/job/32946793308, except the steps related to `pgpkms` (which need AWS credentials, missing from paritytech-stg). The binary names don't have a `darwin-arm` identifier, and conflict with the existing x86_64-linux binaries. I haven't tested building everything on `paritytech-stg` because the x86_64-linux builds run on `unbutu-latest-m` which isn't enabled on `pairtytech-stg` (and I haven't asked CI team to enable one), so testing how to go around naming conflicts should be covered next. - [x] Test the workflow start to end (especially the last bits related to uploading the binaries on S3 and ensuring the previous binaries and the new ones coexist harmoniously on S3/action artifacts storage without naming conflicts) @EgorPopelyaev - [x] Publish the arm binaries on the Github release page - to clarify what's needed @iulianbarbu . Current practice is to manually publish the binaries built via `release-build-rc.yml` workflow, taken from S3. Would be great to have the binaries there in the first place before working on automating this, but I would also do it in a follow up PR. - [ ] unify the binaries building under `release-30_publish_release_draft.yml` maybe? - [ ] automate binary artifacts upload to S3 in `release-30_publish_release_draft.yml` --------- Signed-off-by: Iulian Barbu <[email protected]> Co-authored-by: EgorPopelyaev <[email protected]>
This PR adds the required changes to release `polkadot`, `polkadot-parachain` and `polkadot-omni-node` binaries built on Apple Sillicon macos. This addresses requests from the community for such binaries: #802, and they should be part of the Github release page. Test on paritytech-stg solely focused on macos binaries: https://github.com/paritytech-stg/polkadot-sdk/actions/runs/11824692766/job/32946793308, except the steps related to `pgpkms` (which need AWS credentials, missing from paritytech-stg). The binary names don't have a `darwin-arm` identifier, and conflict with the existing x86_64-linux binaries. I haven't tested building everything on `paritytech-stg` because the x86_64-linux builds run on `unbutu-latest-m` which isn't enabled on `pairtytech-stg` (and I haven't asked CI team to enable one), so testing how to go around naming conflicts should be covered next. - [x] Test the workflow start to end (especially the last bits related to uploading the binaries on S3 and ensuring the previous binaries and the new ones coexist harmoniously on S3/action artifacts storage without naming conflicts) @EgorPopelyaev - [x] Publish the arm binaries on the Github release page - to clarify what's needed @iulianbarbu . Current practice is to manually publish the binaries built via `release-build-rc.yml` workflow, taken from S3. Would be great to have the binaries there in the first place before working on automating this, but I would also do it in a follow up PR. - [ ] unify the binaries building under `release-30_publish_release_draft.yml` maybe? - [ ] automate binary artifacts upload to S3 in `release-30_publish_release_draft.yml` --------- Signed-off-by: Iulian Barbu <[email protected]> Co-authored-by: EgorPopelyaev <[email protected]>
… job (#6427) (#6596) # Description This PR adds the required changes to release `polkadot`, `polkadot-parachain` and `polkadot-omni-node` binaries built on Apple Sillicon macos. ## Integration This addresses requests from the community for such binaries: #802, and they should be part of the Github release page. ## Review Notes Test on paritytech-stg solely focused on macos binaries: https://github.com/paritytech-stg/polkadot-sdk/actions/runs/11824692766/job/32946793308, except the steps related to `pgpkms` (which need AWS credentials, missing from paritytech-stg). The binary names don't have a `darwin-arm` identifier, and conflict with the existing x86_64-linux binaries. I haven't tested building everything on `paritytech-stg` because the x86_64-linux builds run on `unbutu-latest-m` which isn't enabled on `pairtytech-stg` (and I haven't asked CI team to enable one), so testing how to go around naming conflicts should be covered next. ### TODO - [x] Test the workflow start to end (especially the last bits related to uploading the binaries on S3 and ensuring the previous binaries and the new ones coexist harmoniously on S3/action artifacts storage without naming conflicts) @EgorPopelyaev - [x] Publish the arm binaries on the Github release page - to clarify what's needed @iulianbarbu . Current practice is to manually publish the binaries built via `release-build-rc.yml` workflow, taken from S3. Would be great to have the binaries there in the first place before working on automating this, but I would also do it in a follow up PR. ### Follow ups - [ ] unify the binaries building under `release-30_publish_release_draft.yml` maybe? - [ ] automate binary artifacts upload to S3 in `release-30_publish_release_draft.yml` Signed-off-by: Iulian Barbu <[email protected]> Co-authored-by: Iulian Barbu <[email protected]>
We merged recently #6427, which brings support for releasing Apple Silicon ARM node binaries (polkadot, polkadot-prepare/execute-worker, polkadot-parachain & polkadot-omni-node) which will be part of the release pages artifacts section with the next release. The initial request of this issue was for Apple Silicon ARM binaries for development, and the node binaries should be sufficient I think, but please create a more specific issue if something is missing. Other than this, request for ARM Linux binaries was also mentioned (and got some likes), so I opened a dedicated issue here: #6678. |
# Description This PR adds the required changes to release `polkadot`, `polkadot-parachain` and `polkadot-omni-node` binaries built on Apple Sillicon macos. ## Integration This addresses requests from the community for such binaries: paritytech#802, and they should be part of the Github release page. ## Review Notes Test on paritytech-stg solely focused on macos binaries: https://github.com/paritytech-stg/polkadot-sdk/actions/runs/11824692766/job/32946793308, except the steps related to `pgpkms` (which need AWS credentials, missing from paritytech-stg). The binary names don't have a `darwin-arm` identifier, and conflict with the existing x86_64-linux binaries. I haven't tested building everything on `paritytech-stg` because the x86_64-linux builds run on `unbutu-latest-m` which isn't enabled on `pairtytech-stg` (and I haven't asked CI team to enable one), so testing how to go around naming conflicts should be covered next. ### TODO - [x] Test the workflow start to end (especially the last bits related to uploading the binaries on S3 and ensuring the previous binaries and the new ones coexist harmoniously on S3/action artifacts storage without naming conflicts) @EgorPopelyaev - [x] Publish the arm binaries on the Github release page - to clarify what's needed @iulianbarbu . Current practice is to manually publish the binaries built via `release-build-rc.yml` workflow, taken from S3. Would be great to have the binaries there in the first place before working on automating this, but I would also do it in a follow up PR. ### Follow ups - [ ] unify the binaries building under `release-30_publish_release_draft.yml` maybe? - [ ] automate binary artifacts upload to S3 in `release-30_publish_release_draft.yml` --------- Signed-off-by: Iulian Barbu <[email protected]> Co-authored-by: EgorPopelyaev <[email protected]>
Currently, there are only X86 binaries available on the releases page. This is a problem for our team as some people are developing on M1 Macs. Can you please also release ARM binaries?
The text was updated successfully, but these errors were encountered: