Implement Hybrid Functionality and add custom Image resource #413

guineveresaenger · 2022-11-02T21:02:30Z

This pull request restructures this provider to support both TF based and pulumi-native(custom) Resources.
Specifically, this adds Image as an "extra" custom resource to the resources declaration file (resources.go),
and creates a hybrid provider that handles logic to switch between a custom resource and a TF based resource.

Of note

To implement hybrid functionality, the call to tfbridge.Main is circumvented and all RPC methods are implemented directly in hybrid.go.
The logic to distinguish between a custom resource and a TF resource is currently hardcoded via the dockerImageTok variable, as that is the only custom resource currently needed.
The schema contains the foundation for the behavior of Image as it is currently implemented, with the following deviations:
- digest is discontinued as per deprecation message
- id is discontinued, as per deprecation message
- localImageName is discontinued, as users can achieve the same behavior by setting skipPush to true and image tags are lightweight and repeatable.
A dockerBuild method uses the Docker client to start implementing much of the expected (current) behavior of Image:
- Handles user input from the Pulumi program with logic around OneOf (Note that OneOf is not fully supported in all languages so it may make sense to simplify the schema for cacheFrom and build)
- Build image locally
- Push to Registry
- Build context enabled
- SkipPush enabled
- Buildtime args enabled
- Logs build and push output to Pulumi Info
- Errors properly on failure to push and outputs the docker client error
- No separate logic for docker tags - this is a bonus, as the client handles this implicitly via looking for a :mytag at the end of the image name (defaults to :latest)
- TODO: handle registry login for ImagePush to use cacheFrom when set
- TODO: handle setting of env input to translate to setting the BuilderVersion constant in the docker client
- TODO: deprecate/gather usage information on extraOptions as we will need to parse them individually as client options rather can command line flags
- TODO: add behavior for target
- TODO: add behavior for crossplat build

Edited to add Nov.17:

All Docker Image integration tests have been updated and unskipped and are passing (because we have schematized inputs now, several languages will need to migrate from a DockerBuild type to a DockerBuildArgs type).
We have full implementation with Checkpoint objects for Create, Update, and Diff
Read is currently returning only an ID, as a full Read would be a new feature
The unique identifier for the Image resource is imageName - not sure if this is optimal; we may want to extract a build hash here instead.

Fixes #403

provider/provider.go

provider/hybrid.go

t0yv0 · 2022-11-03T14:18:15Z

It looks like the overlays used to be written by hand for every language and with this instead we get automatic extensibility in the languages dimension, that's a huge win!

viveklak · 2022-11-02T21:58:25Z

provider/provider.go

+	if v := req.GetVersion(); v != 0 {
+		return nil, fmt.Errorf("unsupported schema version %d", v)
+	}
+	return &rpc.GetSchemaResponse{Schema: string(p.schemaBytes)}, nil
+}


Nit: I don't think this layer of the GetSchema will ever be called since the schema will be vended by the top-level provider interface. This can probably fail with an assertion here instead.

provider/hybrid.go

viveklak · 2022-11-03T16:41:36Z

provider/hybrid.go

+	tok := urn.Type().String()
+	fmt.Println(tok)
+	// TODO: implement this for actual!!!
+	if tok == dockerImageToken {


Another interesting pattern when it comes to multiplexing custom resource handlers is what is done in the pulumiservice provider: https://github.com/pulumi/pulumi-pulumiservice/blob/main/provider/pkg/provider/provider.go. Essentially you could add a resource interface which could be used to invoke the handlers for each of the supported resources as done in that provider.

github-actions · 2022-11-03T23:33:41Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-03T23:47:22Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

guineveresaenger · 2022-11-04T16:58:58Z

The integration test failures are expected here as the tests need to be updated.

github-actions · 2022-11-09T17:25:52Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-09T17:30:48Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-09T18:09:11Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-09T19:22:03Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-09T19:35:41Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-11T00:45:22Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-11T01:07:06Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-11T01:20:15Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-12T00:48:19Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-14T19:52:01Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-14T21:32:54Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-16T02:25:50Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-16T23:18:19Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-17T01:07:47Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-17T19:50:05Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-17T20:13:06Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-17T21:04:25Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

…provider

…TestDockerfileGo.

github-actions · 2022-11-18T22:45:22Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

github-actions · 2022-11-18T22:47:11Z

Does the PR have any schema changes?

Looking good! No breaking changes found.

New resources:

index/image.Image

docker.DockerBuild was changed to DockerBuildArgs back in Novermber of 2022: pulumi/pulumi-docker#413 (comment)

t0yv0 reviewed Nov 3, 2022

View reviewed changes

provider/provider.go Show resolved Hide resolved

t0yv0 reviewed Nov 3, 2022

View reviewed changes

provider/hybrid.go Show resolved Hide resolved

viveklak reviewed Nov 3, 2022

View reviewed changes

guineveresaenger force-pushed the implement-hybrid branch from b8d219b to ccfc1e1 Compare November 3, 2022 23:30

guineveresaenger mentioned this pull request Nov 5, 2022

Implement Docker Image custom resource functionality #404

Closed

guineveresaenger mentioned this pull request Nov 14, 2022

Add missing await statement in docker.Image Python implementation. #408

Closed

guineveresaenger changed the title ~~Implement Hybrid Functionality~~ Implement Hybrid Functionality and add custom Image resource Nov 14, 2022

guineveresaenger marked this pull request as ready for review November 14, 2022 20:12

guineveresaenger force-pushed the implement-hybrid branch from f9779a0 to 2898ead Compare November 14, 2022 21:29

guineveresaenger added 21 commits November 18, 2022 14:35

Make the URN token for Image be a constant

3b9b320

Implement Diff

f52d3dd

Add an ID field for the resource using imageName

76ee778

Fix Checkpoint logic for Diff, Create, and Update. Enable Secrets on …

8ed1b08

…provider

Unskip tests

e069fb5

linter fixes

1e99ede

Fix Dotnet type. Fix Go version.

56c0478

update go veersion in examples folder. Update package dependency for …

c36c9c9

…TestDockerfileGo.

DockerBuild is now DockerBuildArgs

4791f79

Add SkipNulls:true

252dc60

skip dotnet test for now

661fc2e

Unskip test; require Pulumi.Docker.Inputs

43d0178

add a semicolon

78f29ba

Remove test skip line

15a5029

remove TODOs and add hybrid provider context

98eea7f

set Registry in a separate function

4277ab9

make Password secret

ada7fdd

do not panic

d1bc212

remove duplicate call to Close(); pass along existing Context

924b556

Remove boilerplate cruft

86b7975

update azure-py example so it at least compiles

57a0ef0

guineveresaenger force-pushed the implement-hybrid branch from fb7beee to 57a0ef0 Compare November 18, 2022 22:41

server not registry in registryargs

a81341e

guineveresaenger merged commit d58ed38 into master Nov 18, 2022

pulumi-bot deleted the implement-hybrid branch November 18, 2022 23:04

guineveresaenger mentioned this pull request Nov 18, 2022

Migrate to new Image Resource #405

Closed

maravedi added a commit to maravedi/examples that referenced this pull request Aug 30, 2023

Update __main__.py

bfb8ae3

docker.DockerBuild was changed to DockerBuildArgs back in Novermber of 2022: pulumi/pulumi-docker#413 (comment)

maravedi mentioned this pull request Aug 30, 2023

Update __main__.py pulumi/examples#1493

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Hybrid Functionality and add custom Image resource #413

Implement Hybrid Functionality and add custom Image resource #413

guineveresaenger commented Nov 2, 2022 •

edited

Loading

t0yv0 commented Nov 3, 2022

viveklak Nov 2, 2022

viveklak Nov 3, 2022

github-actions bot commented Nov 3, 2022

github-actions bot commented Nov 3, 2022

guineveresaenger commented Nov 4, 2022

github-actions bot commented Nov 9, 2022

github-actions bot commented Nov 9, 2022

github-actions bot commented Nov 9, 2022

github-actions bot commented Nov 9, 2022

github-actions bot commented Nov 9, 2022

github-actions bot commented Nov 11, 2022

github-actions bot commented Nov 11, 2022

github-actions bot commented Nov 11, 2022

github-actions bot commented Nov 12, 2022

github-actions bot commented Nov 14, 2022

github-actions bot commented Nov 14, 2022

github-actions bot commented Nov 16, 2022

github-actions bot commented Nov 16, 2022

github-actions bot commented Nov 17, 2022

github-actions bot commented Nov 17, 2022

github-actions bot commented Nov 17, 2022

github-actions bot commented Nov 17, 2022

github-actions bot commented Nov 18, 2022

github-actions bot commented Nov 18, 2022

Implement Hybrid Functionality and add custom Image resource #413

Implement Hybrid Functionality and add custom Image resource #413

Conversation

guineveresaenger commented Nov 2, 2022 • edited Loading

t0yv0 commented Nov 3, 2022

viveklak Nov 2, 2022

Choose a reason for hiding this comment

viveklak Nov 3, 2022

Choose a reason for hiding this comment

github-actions bot commented Nov 3, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 3, 2022

Does the PR have any schema changes?

New resources:

guineveresaenger commented Nov 4, 2022

github-actions bot commented Nov 9, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 9, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 9, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 9, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 9, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 11, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 11, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 11, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 12, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 14, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 14, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 16, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 16, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 17, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 17, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 17, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 17, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 18, 2022

Does the PR have any schema changes?

New resources:

github-actions bot commented Nov 18, 2022

Does the PR have any schema changes?

New resources:

guineveresaenger commented Nov 2, 2022 •

edited

Loading