Add parameter base_image and addon_image to BUILD_PLATFORMS

[mauriciopoppe]

Fixes #145

Adds new parameters in BUILD_PLATFORMS referred as base_image and addon_image with the format image:tag

make push-multiarch BUILD_PLATFORMS="linux amd64; windows amd64 .exe nanoserver:2019 servercore:2019;

List of changes:

• made /bin/bash the default shell instead of /bin/sh, I needed to do this to do string addition and indexOf
• updated the cloudbuild docker image to a newer version that has 20.10 and this fix: docker manifest annotate should be able to add os.version to an image(Windows version support) docker/cli#2722 (comment)
• docker build with multiple build platforms with different base images
• created a docker manifest with info about all of these builds including info about the combinations of arch, os and os-version

Output of some snippets used here:

build_platforms=build_platforms="linux amd64; linux ppc64le -ppc64le; linux s390x -s390x; linux arm64 -arm64; windows amd64 .exe nanoserver:1809 servercore:ltsc2019; windows amd64 .exe nanoserver:1909 servercore:1909; windows amd64 .exe nanoserver:2004 servercore:2004; windows amd64 .exe nanoserver:20H2 servercore:20H2"
echo "$build_platforms" | sed -e 's/windows *[^ ]* *.exe *[^ ]* *[^ ]*//g' -e 's/; *;/;/g' -e 's/;[ ]*$//'
linux amd64; linux ppc64le -ppc64le; linux s390x -s390x; linux arm64 -arm64

I used these changes in node-driver-registrar to push all the windows images as well as the manifest:

Manifest contents of the canary manifest:

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 738,
         "digest": "sha256:619f0eb76b5b704fe351dacc26b94368e7e0db60a1fefcd8605aeb3e7e81874e",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 950,
         "digest": "sha256:5a21cee79643052c985a0d7073bef60f36711c83d74f58e94605c3e8a1ae6abf",
         "platform": {
            "architecture": "amd64",
            "os": "windows",
            "os.version": "10.0.17763.1935"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 950,
         "digest": "sha256:b5f11b3fe645a67b3ae2ab3bf32403670c6c454c4ab19e72a41c052035613873",
         "platform": {
            "architecture": "amd64",
            "os": "windows",
            "os.version": "10.0.18363.1556"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 950,
         "digest": "sha256:fe8327a9e9d3c2c73fa117cb4b635fc2c4f3f501e217ffe4f36bb6918cb63b3e",
         "platform": {
            "architecture": "amd64",
            "os": "windows",
            "os.version": "10.0.19041.985"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 950,
         "digest": "sha256:fe57e7ddfcec7008290dbf998c131ac5d98c8514326555d8fb24607b17e7ca0a",
         "platform": {
            "architecture": "amd64",
            "os": "windows",
            "os.version": "10.0.19042.985"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 738,
         "digest": "sha256:cd870ccc4e58a592195e3af0cc3e690cc009a6fce7a47deaedc7583ccc335310",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 738,
         "digest": "sha256:56e373d03c5a0e00d8e09912d462984bd5169433fca887505310f4b179430386",
         "platform": {
            "architecture": "ppc64le",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 738,
         "digest": "sha256:a11755a59d704d2da641794aa31235875f1d2b65d24810831b952220fcdec86f",
         "platform": {
            "architecture": "s390x",
            "os": "linux"
         }
      }
   ]
}

/assign @pohly @jingxu97

[pohly]

I think you can squash all of this into one commit.

[jingxu97]

could you give me an example about this logic?

[mauriciopoppe]

Sure, so if you're iterating over this array of tuples: linux amd64; linux ppc64le -ppc64le; linux s390x -s390x; linux arm64 -arm64; windows amd64 .exe nanoserver:1809 servercore:ltsc2019; windows amd64 .exe nanoserver:1909 servercore:1909; windows amd64 .exe nanoserver:2004 servercore:2004; windows amd64 .exe nanoserver:20H2 servercore:20H2

go build will be called like this:

+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=amd64
+ go build -a -ldflags ' -X main.version=v12345 -extldflags "-static"' -o ./bin/csi-node-driver-registrar ./cmd/csi-node-driver-registrar
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=ppc64le
+ go build -a -ldflags ' -X main.version=v12345 -extldflags "-static"' -o ./bin/csi-node-driver-registrar-ppc64le ./cmd/csi-node-driver-registrar
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=s390x
+ go build -a -ldflags ' -X main.version=v12345 -extldflags "-static"' -o ./bin/csi-node-driver-registrar-s390x ./cmd/csi-node-driver-registrar
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=arm64
+ go build -a -ldflags ' -X main.version=v12345 -extldflags "-static"' -o ./bin/csi-node-driver-registrar-arm64 ./cmd/csi-node-driver-registrar
+ CGO_ENABLED=0
+ GOOS=windows
+ GOARCH=amd64
+ go build -a -ldflags ' -X main.version=v12345 -extldflags "-static"' -o ./bin/csi-node-driver-registrar.exe ./cmd/csi-node-driver-registrar
+ CGO_ENABLED=0
+ GOOS=windows
+ GOARCH=amd64
+ go build -a -ldflags ' -X main.version=v12345 -extldflags "-static"' -o ./bin/csi-node-driver-registrar.exe ./cmd/csi-node-driver-registrar
+ CGO_ENABLED=0
+ GOOS=windows
+ GOARCH=amd64
+ go build -a -ldflags ' -X main.version=v12345 -extldflags "-static"' -o ./bin/csi-node-driver-registrar.exe ./cmd/csi-node-driver-registrar
+ CGO_ENABLED=0
+ GOOS=windows
+ GOARCH=amd64
+ go build -a -ldflags ' -X main.version=v12345 -extldflags "-static"' -o ./bin/csi-node-driver-registrar.exe ./cmd/csi-node-driver-registrar

As you can see the windows binary is built 4 times for the same os,arch combination, with this code I'm making sure that we only build it once per os,arch combination

+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=amd64
+ go build -a -ldflags ' -X main.version=v12345 -extldflags "-static"' -o ./bin/csi-node-driver-registrar ./cmd/csi-node-driver-registrar
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=ppc64le
+ go build -a -ldflags ' -X main.version=v12345 -extldflags "-static"' -o ./bin/csi-node-driver-registrar-ppc64le ./cmd/csi-node-driver-registrar
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=s390x
+ go build -a -ldflags ' -X main.version=v12345 -extldflags "-static"' -o ./bin/csi-node-driver-registrar-s390x ./cmd/csi-node-driver-registrar
+ CGO_ENABLED=0
+ GOOS=linux
+ GOARCH=arm64
+ go build -a -ldflags ' -X main.version=v12345 -extldflags "-static"' -o ./bin/csi-node-driver-registrar-arm64 ./cmd/csi-node-driver-registrar
+ CGO_ENABLED=0
+ GOOS=windows
+ GOARCH=amd64
+ go build -a -ldflags ' -X main.version=v12345 -extldflags "-static"' -o ./bin/csi-node-driver-registrar.exe ./cmd/csi-node-driver-registrar

[pohly]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mauriciopoppe, pohly

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [pohly]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mauriciopoppe]

@pohly I made a small change to the name of the image, from amd64-nanoserver-20H2-windows-canary to amd64-windows-nanoserver-20H2-canary, the lgtm tag was removed because of this

[mauriciopoppe]

@pohly so sorry I lost your comment, are the images in csi-driver-host-path still failing because of this?

[mauriciopoppe]

@pohly I just checked that you added the missing $ in 5561780, thank you and sorry about missing this.