Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ops,ci: docker buildx bake, fix publish/release flows #7811

Merged
merged 2 commits into from
Oct 25, 2023

Conversation

protolambda
Copy link
Contributor

@protolambda protolambda commented Oct 24, 2023

Description

In #7715 we changed the docker build system to build the following modules with a shared docker image, to reduce CI work and improve cache usage:

  • op-heartbeat
  • op-challenger
  • op-node
  • op-batcher
  • op-proposer
  • op-program
  • op-wheel

Most of these we also publish (everything except op-wheel), and release (everything except op-program and op-wheel).

However, we were building the same docker images with 3 different job definitions that then had to correctly invoke the docker build.

To try and fix that, I unified the docker_build / docker_publish / docker_release jobs, since most of the parameters and work overlaps.

With new publish and release job parameters (boolean flags) the job can do the steps for publishing/releasing conditionally.

And to reduce the CI configuration more, I moved the docker configuration of all the targets into a docker-bake setup (see docker-bake.hcl): see make golang-docker for an example that builds the common op-node op-batcher op-proposer op-challenger targets.

@protolambda protolambda requested a review from a team as a code owner October 24, 2023 14:34
.circleci/config.yml Outdated Show resolved Hide resolved
.circleci/config.yml Outdated Show resolved Hide resolved
Copy link
Collaborator

@mslipper mslipper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@trianglesphere trianglesphere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CI workflow is now failing. I'm not sure why, but it's in the op-stack-go-docker-build-1 step and looks like it's caused by these changes.

.circleci/config.yml Show resolved Hide resolved
@protolambda protolambda changed the title ci: unify docker build/publish/release, update publish/release flows for shared go stack builder ops,ci: docker buildx bake, fix publish/release flows Oct 25, 2023
@protolambda protolambda requested a review from mslipper October 25, 2023 13:40
@protolambda protolambda force-pushed the fix-docker-releases branch 4 times, most recently from 67c36d6 to 6856420 Compare October 25, 2023 14:00
@protolambda protolambda force-pushed the fix-docker-releases branch 2 times, most recently from 8ac6b6b to de78eb2 Compare October 25, 2023 15:15
@tynes
Copy link
Contributor

tynes commented Oct 25, 2023

As long as we are ok with depending on an experimental feature, i am cool with this
https://docs.docker.com/build/bake/

@tynes tynes enabled auto-merge October 25, 2023 18:28
@tynes tynes added this pull request to the merge queue Oct 25, 2023
@tynes tynes removed this pull request from the merge queue due to a manual request Oct 25, 2023
@tynes tynes merged commit 536b44b into develop Oct 25, 2023
4 checks passed
@tynes tynes deleted the fix-docker-releases branch October 25, 2023 18:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants