Migrating existing stack using pulumi docker 3.1.0 to 4.0.1 fails with `error: expected '::' in provider reference ''` #563

briandestura · 2023-03-22T00:18:40Z

What happened?

I have a stack that has images built using pulumi docker v3.1.0. Ive done the changes to migrate to v4.1.0 and tried to update my stack, but I get this error:

Step #3 - "run-pulumi": ~ pulumi:pulumi:Stack: (refresh)
Step #3 - "run-pulumi":     [urn=urn:pulumi:staging::my-project::pulumi:pulumi:Stack::my-project-staging]
Step #3 - "run-pulumi":   pulumi:pulumi:Stack: (same)
Step #3 - "run-pulumi":     [urn=urn:pulumi:staging::my-project::pulumi:pulumi:Stack::my-project-staging]
Step #3 - "run-pulumi": error: expected '::' in provider reference ''

Running just pulumi preview raises the same error message. From a slack conversation I've got to know it's got something to do with v3 not requiring providers for docker images.

A workaround is to delete those images from the statefile and rebuild

Expected Behavior

V4 release notes suggest code changes are all that's needed so was expecting it to work outright

Steps to reproduce

Build a stack with images using pulumi docker v.3.1.0
Upgrade pulumi docker to v.4.0.1 along with the required code changes and run pulumi refresh

Output of `pulumi about`

11:15:01 > $ pipenv run pulumi about
CLI
Version      3.58.0
Go Version   go1.20.1
Go Compiler  gc

Plugins
NAME    VERSION
docker  4.0.1
gcp     6.51.0
python  unknown

Host
OS       darwin
Version  13.0.1
Arch     arm64


Dependencies:
NAME           VERSION
pip            23.0.1
pulumi-docker  4.0.1
pulumi-gcp     6.51.0
setuptools     67.6.0
wheel          0.38.4

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

The text was updated successfully, but these errors were encountered:

t0yv0 · 2023-03-22T15:38:11Z

Thank you for reporting!

Could you help us out here with a bit of a more detailed repro (GitHub gist or repo is ideal)? Curious whether any resource options are used on the Image resource etc. Custom first-class provider perhaps?

Build a stack with images using pulumi docker v.3.1.0

thekarel · 2023-03-27T11:15:58Z

I also had to delete the v3 image and then recreate it with v4.

v3 Image definition:

  const dockerImage = new docker.Image('project-app', {
    imageName: `eu.gcr.io/project/app:latest`,
    build: {
      context: '../container/',
      dockerfile: './Dockerfile',
    },
  })

For what it's worth, the diff is

v3 Image

  - dockerImage        : {
      - baseImageName: "eu.gcr.io/PROJECT-dev/PROJECT-trpc:latest"
      - digest       : "eu.gcr.io/PROJECT-dev/PROJECT-trpc:latest-HASH"
      - id           : "eu.gcr.io/PROJECT-dev/PROJECT-trpc:latest-HASH"
      - imageName    : "eu.gcr.io/PROJECT-dev/PROJECT-trpc:latest-HASH"
      - urn          : "urn:pulumi:dev::PROJECT-trpc::docker:image:Image::PROJECT-trpc"
    }

v4 Image

+  docker:index/image:Image (create)
    [urn=urn:pulumi:dev::PROJECT-trpc::docker:index/image:Image::PROJECT-trpc]
    build    : {
        builderVersion: "BuilderBuildKit"
        cacheFrom     : {
            images: [
                [0]: "eu.gcr.io/PROJECT-dev/PROJECT-trpc:latest"
            ]
        }
        context       : "../container/"
        contextDigest : "HASH"
        dockerfile    : "./Dockerfile"
        platform      : "linux/amd64"
    }
    imageName: "eu.gcr.io/PROJECT-dev/PROJECT-trpc:latest"
    skipPush : false

❯ pulumi about
CLI
Version      3.59.1
Go Version   go1.20.2
Go Compiler  gc

Plugins
NAME    VERSION
nodejs  unknown

Host
OS       darwin
Version  13.2.1
Arch     x86_64

This project is written in nodejs: executable='/Users/x/.volta/bin/node' version='v18.14.2'

guineveresaenger · 2023-03-27T18:39:36Z

Hi @briandestura - I have successfully reproduced this with 3.1.0 --> 4.0.0 as well as 3.6.1 --> 4.0.0. Thank you for flagging this!

Because Image was not a true Stack resource in v 3.x.x, the change in URN from
docker::docker:image:Image
to
docker::docker:index/image:Image

will result in a failure to update. The error message itself is somewhat misleading; a transformation of a non-schematized resource to a fully indexed one is a first for us, and we appreciate your patience here!

UPDATE We did alias the old resource URN with exactly this scenario in mind; so it appears this is a real bug that should not occur. Please stay tuned for a real fix.
Because the previous Image Resouce did not instantiate a default provider, during the upgrade, Pulumi will look for a provider associated with the Image resource, but as there isn't one, it will fail. We are looking into this further. In the meantime, the workaround below is hopefully helpful.

I believe removing the old Image resource from your stack file via pulumi state delete or alternately via pulumi stack export/import (details below) and re-running pulumi up will be the path forward. I will verify with the team. I will also add the steps to resolve to the migration guide.

Details for resolution

via state delete

pulumi stack --show-urns
pulumi state delete <any Image URN> <--- this is safe to do, as Images have no cloud state
upgrade to 4.x.x
pulumi up

via stack import/export

Export your Stack state to a JSON file: pulumi stack export > stack-file.json
Delete any Image Resources using the old URN of docker::docker:image:Image from the stack file. It could look something like this:

            {
                "urn": "urn:pulumi:dev::docker::docker:image:Image::my-image",
                "custom": false,
                "type": "docker:image:Image",
                "outputs": {
                    "baseImageName": "gsaenger/demo-image:latest",
                    "digest": "gsaenger/demo-image:latest-9e5aea46035631f0d0a9b681eb72f427289bf467f3f1dde3c816980639cabe5e",
                    "id": "gsaenger/demo-image:latest-9e5aea46035631f0d0a9b681eb72f427289bf467f3f1dde3c816980639cabe5e",
                    "imageName": "gsaenger/demo-image:latest-9e5aea46035631f0d0a9b681eb72f427289bf467f3f1dde3c816980639cabe5e",
                    "registryServer": "docker.io"
                },
                "parent": "urn:pulumi:dev::docker::pulumi:pulumi:Stack::docker-dev"
            },

Import the updated Stack: pulumi stack import --file stack-file.json
Re-run pulumi up with the new Docker version.

guineveresaenger · 2023-03-27T18:47:49Z

Migration guide updated here: https://github.com/pulumi/pulumi-docker/releases/tag/v4.0.0

briandestura · 2023-03-28T00:00:53Z

Thank you @guineveresaenger ! Also apologies @t0yv0 , got busy with personal stuff so didn't have time to add a detailed repro

guineveresaenger · 2023-03-28T16:50:52Z

Update - I discovered there was an easier workaround via pulumi state delete which allows you to remove individual resources from a stack a bit more easily than manually editing a JSON file 🎉. See edit above.

guineveresaenger · 2023-03-29T17:42:07Z

Final update: I spoke with my colleague @Frassle and this turns out to be a bug in the Pulumi engine. I have filed an issue there. Once a fix is made and released, all you will need for a seamless upgrade will be to update your pulumi version.

I will leave this issue open but blocked on Pulumi engine work. Please use the workarounds described in the meantime.

guineveresaenger · 2023-03-30T16:59:20Z

We have a fix via pulumi/pulumi#12561. On your next pulumi version upgrade, the pulujmi-docker upgrade should work as expected.

briandestura added kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels Mar 22, 2023

t0yv0 removed the needs-triage Needs attention from the triage team label Mar 22, 2023

t0yv0 added the needs-repro Needs repro steps before it can be triaged or fixed label Mar 22, 2023

guineveresaenger added resolution/by-design This issue won't be fixed because the functionality is working as designed and removed needs-repro Needs repro steps before it can be triaged or fixed labels Mar 27, 2023

guineveresaenger self-assigned this Mar 27, 2023

guineveresaenger added p1 A bug severe enough to be the next item assigned to an engineer and removed resolution/by-design This issue won't be fixed because the functionality is working as designed labels Mar 27, 2023

guineveresaenger mentioned this issue Mar 29, 2023

Upgrading a component resource to a custom resource fails pulumi/pulumi#12550

Closed

guineveresaenger removed the p1 A bug severe enough to be the next item assigned to an engineer label Mar 29, 2023

guineveresaenger closed this as completed Mar 30, 2023

AaronFriel added this to the 0.86 milestone Apr 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrating existing stack using pulumi docker 3.1.0 to 4.0.1 fails with `error: expected '::' in provider reference ''` #563

Migrating existing stack using pulumi docker 3.1.0 to 4.0.1 fails with `error: expected '::' in provider reference ''` #563

briandestura commented Mar 22, 2023 •

edited

Loading

t0yv0 commented Mar 22, 2023 •

edited

Loading

thekarel commented Mar 27, 2023 •

edited

Loading

guineveresaenger commented Mar 27, 2023 •

edited

Loading

guineveresaenger commented Mar 27, 2023

briandestura commented Mar 28, 2023

guineveresaenger commented Mar 28, 2023

guineveresaenger commented Mar 29, 2023

guineveresaenger commented Mar 30, 2023

Migrating existing stack using pulumi docker 3.1.0 to 4.0.1 fails with error: expected '::' in provider reference '' #563

Migrating existing stack using pulumi docker 3.1.0 to 4.0.1 fails with error: expected '::' in provider reference '' #563

Comments

briandestura commented Mar 22, 2023 • edited Loading

What happened?

Expected Behavior

Steps to reproduce

Output of pulumi about

Additional context

Contributing

t0yv0 commented Mar 22, 2023 • edited Loading

thekarel commented Mar 27, 2023 • edited Loading

v3 Image definition:

v3 Image

v4 Image

guineveresaenger commented Mar 27, 2023 • edited Loading

Details for resolution

via state delete

via stack import/export

guineveresaenger commented Mar 27, 2023

briandestura commented Mar 28, 2023

guineveresaenger commented Mar 28, 2023

guineveresaenger commented Mar 29, 2023

guineveresaenger commented Mar 30, 2023

Migrating existing stack using pulumi docker 3.1.0 to 4.0.1 fails with `error: expected '::' in provider reference ''` #563

Migrating existing stack using pulumi docker 3.1.0 to 4.0.1 fails with `error: expected '::' in provider reference ''` #563

briandestura commented Mar 22, 2023 •

edited

Loading

Output of `pulumi about`

t0yv0 commented Mar 22, 2023 •

edited

Loading

thekarel commented Mar 27, 2023 •

edited

Loading

guineveresaenger commented Mar 27, 2023 •

edited

Loading