Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Process and display build and push logs #450

Merged
merged 15 commits into from
Jan 19, 2023
Merged

Process and display build and push logs #450

merged 15 commits into from
Jan 19, 2023

Conversation

guineveresaenger
Copy link
Contributor

@guineveresaenger guineveresaenger commented Jan 4, 2023 •
edited
Loading

This PR leverages Pulumi's host.LogStatus function to display Docker image logs "ephemerally", i.e. in the "Info" status column on the Pulumi terminal output.
Additionally, this change uses Docker's JSONMessage package to parse the information given by the Docker build process.
Using this package allows us to uniformly process all logs sent by Docker and have reasonable baseline expectations for informational fields.
The local implementation leans strongly on the logic in jsonmessage.Display to extract all relevant information.
The result is that the user can watch Docker build logs in the Pulumi Info box.

Additional changes made:

  • Adds a Buildkit Session for pushing a Docker image with BuildKit to avoid an error where Buildkit pulls from a registry for base images if a local image is not available - but it can only do so with an active buildkit session.
  • Uses github.com/moby/buildkit/api/services/control to parse and prettify the Buildkit tracer logs, which are both encrypted and all sent to jsonmessage's aux field, as well as encrypted.

Fixes #441.

@github-actions
Copy link

github-actions bot commented Jan 4, 2023

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

@guineveresaenger
Copy link
Contributor Author

guineveresaenger commented Jan 4, 2023 •
edited
Loading

Here's a little screen recording:

asciicast

@github-actions
Copy link

github-actions bot commented Jan 4, 2023

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

@guineveresaenger guineveresaenger requested a review from a team January 4, 2023 20:42
@github-actions
Copy link

github-actions bot commented Jan 4, 2023

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

@github-actions
Copy link

github-actions bot commented Jan 5, 2023

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

@github-actions
Copy link

github-actions bot commented Jan 5, 2023

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

@github-actions
Copy link

github-actions bot commented Jan 7, 2023

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

t0yv0
t0yv0 reviewed Jan 10, 2023
View reviewed changes
provider/image.go Outdated Show resolved Hide resolved
t0yv0
t0yv0 reviewed Jan 10, 2023
View reviewed changes
provider/image.go Outdated Show resolved Hide resolved
t0yv0
t0yv0 reviewed Jan 10, 2023
View reviewed changes
provider/image.go Outdated Show resolved Hide resolved
t0yv0
t0yv0 reviewed Jan 10, 2023
View reviewed changes
provider/image.go Outdated Show resolved Hide resolved
t0yv0
t0yv0 reviewed Jan 10, 2023
View reviewed changes
provider/image.go Outdated Show resolved Hide resolved
t0yv0
t0yv0 reviewed Jan 10, 2023
View reviewed changes
provider/image.go Outdated Show resolved Hide resolved
t0yv0
t0yv0 reviewed Jan 10, 2023
View reviewed changes
Copy link
Member

@t0yv0 t0yv0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! I have a few small nits but the biggest question is what's the goroutine doing? It just smells like a race to me but perhaps it isn't - I did not dive into the underlying API too much.

@t0yv0 t0yv0 self-requested a review January 18, 2023 14:30
@guineveresaenger
Add buildkit parsing for jsonmessage.Aux field
fb2e25c 
Buildkit sends build logs with the id of `moby.buildkit.trace` to the
`aux` field of jsonmessage.
This commit adds logic akin to that in
`https://github.com/docker/docker-ce` to properly decode and parse the
buildkit build logs.
@github-actions
Copy link

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

@github-actions
Copy link

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

@github-actions
Copy link

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

t0yv0
t0yv0 approved these changes Jan 18, 2023
View reviewed changes
Copy link
Member

@t0yv0 t0yv0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like https://github.com/moby/buildkit/blob/b78713cdd127b359b40ae0658c36cbab1ed28d60/session/session.go#L135 s.Close waits for termination.

@t0yv0
Copy link
Member

t0yv0 commented Jan 18, 2023

Wanted to ask for tests here but since it's log redirection feature perhaps OK to elide. Was wondering about

Working on this PR exposed a bug?missing feature? for running the Docker SDK in buildkit mode.

The buildkit support is covered by existing test suite yes?

@guineveresaenger
Copy link
Contributor Author

Yes - the default is now Buildkit, meaning our tests will build images in Buildkit mode. 👍

@guineveresaenger guineveresaenger merged commit dedc353 into master Jan 19, 2023
@guineveresaenger guineveresaenger deleted the ephemeral branch January 19, 2023 01:11
@mikhailshilkov mikhailshilkov mentioned this pull request Oct 31, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@t0yv0 t0yv0 t0yv0 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Log output parsing
3 participants
@guineveresaenger @t0yv0 @AaronFriel