Skip to content

Commit

Permalink
Update Moby dependencies, address skipped builds (#828)
Browse files Browse the repository at this point in the history
Adding additional logging to `sess.Run()` call here, and fixing
moby/moby#45935, as a prerequisite to debugging #812.

While looking into #812, when using a locally built provider, updates to
the Image resource would not result in a new image built and a
corresponding new `RepoDigest` output value. This was root caused to the
Moby issue below after adding a log statement following `sess.Run()` in
`provider/image.go`.
- moby/moby#45935

However, the behavior seen while reproducing is *different* from our
issue. That local repro was too consistent, it occurred in over 20
consecutive builds following the repro steps at the bottom of this
commit.

After updating the Moby dependencies, the image build was no longer
skipped and I was unable to reproduce #812. Neither [Docker v23 release
notes](https://docs.docker.com/engine/release-notes/23.0/) nor the [git
diff from 23.0.1 to
23.0.7](moby/moby@v23.0.1...v23.0.7) contain
any obvious changes that explain that.

The most likely conclusion here is:
1. There is still an unknown bug, likely a race condition, that can
trigger issue 812, and I was unsuccessful in reproducing it.
2. The Moby dependency update fixed an issue with the session dialing
that has the same symptoms as issue 812, which is fixed in the change to
`client/client.go` in the Moby diff above.

Repro
=====

The Pulumi program in `examples/multi-stage-build-go` was run repeatedly
to trigger #812 via:

```console
$ uuidgen -r > ./app/example-lockfile.json && pulumi up --skip-preview --diff
```

When the symptoms occurred, verbose logging indicated the image build
was skipped entirely - `docker.ImageBuild` did not run.
  • Loading branch information
AaronFriel authored Nov 13, 2023
1 parent 06a0a9f commit 7284bef
Show file tree
Hide file tree
Showing 3 changed files with 15 additions and 11 deletions.
6 changes: 3 additions & 3 deletions provider/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ module github.com/pulumi/pulumi-docker/provider/v4
go 1.21

require (
github.com/docker/cli v23.0.1+incompatible
github.com/docker/cli v23.0.7+incompatible
github.com/docker/distribution v2.8.1+incompatible
github.com/docker/docker v23.0.1+incompatible
github.com/docker/docker v23.0.7+incompatible
github.com/golang/protobuf v1.5.3
github.com/moby/buildkit v0.11.4
github.com/moby/moby v23.0.3+incompatible
github.com/moby/moby v23.0.7+incompatible
github.com/pkg/errors v0.9.1
github.com/pulumi/pulumi-terraform-bridge/v3 v3.63.2
github.com/pulumi/pulumi/pkg/v3 v3.91.1
Expand Down
12 changes: 6 additions & 6 deletions provider/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -1121,8 +1121,8 @@ github.com/dnaeon/go-vcr v1.1.0/go.mod h1:M7tiix8f0r6mKKJ3Yq/kqU1OYf3MnfmBWVbPx/
github.com/dnaeon/go-vcr v1.2.0 h1:zHCHvJYTMh1N7xnV7zf1m1GPBF9Ad0Jk/whtQ1663qI=
github.com/dnaeon/go-vcr v1.2.0/go.mod h1:R4UdLID7HZT3taECzJs4YgbbH6PIGXB6W/sc5OLb6RQ=
github.com/docker/cli v0.0.0-20191017083524-a8ff7f821017/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/cli v23.0.1+incompatible h1:LRyWITpGzl2C9e9uGxzisptnxAn1zfZKXy13Ul2Q5oM=
github.com/docker/cli v23.0.1+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/cli v23.0.7+incompatible h1:b+cDxRx/f6P1JR6asBXJOneubySm4nE8CB1B/3g59Sw=
github.com/docker/cli v23.0.7+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/distribution v0.0.0-20190905152932-14b96e55d84c/go.mod h1:0+TTO4EOBfRPhZXAeF1Vu+W3hHZ8eLp8PgKVZlcvtFY=
github.com/docker/distribution v2.7.1-0.20190205005809-0d3efadf0154+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
Expand All @@ -1131,8 +1131,8 @@ github.com/docker/distribution v2.8.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4Kfc
github.com/docker/docker v1.4.2-0.20190924003213-a8608b5b67c7/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v20.10.14+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v20.10.21+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v23.0.1+incompatible h1:vjgvJZxprTTE1A37nm+CLNAdwu6xZekyoiVlUZEINcY=
github.com/docker/docker v23.0.1+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v23.0.7+incompatible h1:6RD6ZfR9LYacFt90L41R/6/SyO+RZ1zWU7+bt6tdRNk=
github.com/docker/docker v23.0.7+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker-credential-helpers v0.6.3/go.mod h1:WRaJzqw3CTB9bk10avuGsjVBZsD05qeibJ1/TYlvc0Y=
github.com/docker/docker-credential-helpers v0.7.0 h1:xtCHsjxogADNZcdv1pKUHXryefjlVRqWqIhk/uXJp0A=
github.com/docker/docker-credential-helpers v0.7.0/go.mod h1:rETQfLdHNT3foU5kuNkFR1R1V12OJRRO5lzt2D1b5X0=
Expand Down Expand Up @@ -1978,8 +1978,8 @@ github.com/moby/buildkit v0.11.4 h1:mleVHr+n7HUD65QNUkgkT3d8muTzhYUoHE9FM3Ej05s=
github.com/moby/buildkit v0.11.4/go.mod h1:P5Qi041LvCfhkfYBHry+Rwoo3Wi6H971J2ggE+PcIoo=
github.com/moby/locker v1.0.1 h1:fOXqR41zeveg4fFODix+1Ch4mj/gT0NE1XJbp/epuBg=
github.com/moby/locker v1.0.1/go.mod h1:S7SDdo5zpBK84bzzVlKr2V0hz+7x9hWbYC/kq7oQppc=
github.com/moby/moby v23.0.3+incompatible h1:g+qvdrXfskgMXiq6DQMo5EdEIp8mo/9befVIddWstGI=
github.com/moby/moby v23.0.3+incompatible/go.mod h1:fDXVQ6+S340veQPv35CzDahGBmHsiclFwfEygB/TWMc=
github.com/moby/moby v23.0.7+incompatible h1:g0V6QLBeYUcZq4tSHdzkTXQCxIs4PvxItxRo27sVqR0=
github.com/moby/moby v23.0.7+incompatible/go.mod h1:fDXVQ6+S340veQPv35CzDahGBmHsiclFwfEygB/TWMc=
github.com/moby/patternmatcher v0.5.0 h1:YCZgJOeULcxLw1Q+sVR636pmS7sPEn1Qo2iAN6M7DBo=
github.com/moby/patternmatcher v0.5.0/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc=
github.com/moby/spdystream v0.2.0/go.mod h1:f7i0iNDQJ059oMTcWxx8MA/zKFIuD/lY+0GqbN2Wy8c=
Expand Down
8 changes: 6 additions & 2 deletions provider/image.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ import (
"github.com/docker/docker/pkg/jsonmessage"
structpb "github.com/golang/protobuf/ptypes/struct"
controlapi "github.com/moby/buildkit/api/services/control"
"github.com/moby/buildkit/identity"
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/session/auth/authprovider"
"github.com/moby/moby/registry"
Expand Down Expand Up @@ -253,7 +254,10 @@ func (p *dockerNativeProvider) dockerBuild(ctx context.Context,

//Start a session for BuildKit
if build.BuilderVersion == defaultBuilder {
sess, _ := session.NewSession(ctx, "pulumi-docker", "")
sess, err := session.NewSession(ctx, "pulumi-docker", identity.NewID())
if err != nil {
return "", nil, err
}

dockerAuthProvider := authprovider.NewDockerAuthProvider(cfg)
sess.Allow(dockerAuthProvider)
Expand All @@ -264,7 +268,7 @@ func (p *dockerNativeProvider) dockerBuild(ctx context.Context,
go func() {
err := sess.Run(ctx, dialSession)
if err != nil {
return
_ = p.host.Log(ctx, "error", urn, fmt.Sprintf("Error running BuildKit session: %v", err))
}
}()
defer sess.Close()
Expand Down

0 comments on commit 7284bef

Please sign in to comment.