-
Notifications
You must be signed in to change notification settings - Fork 212
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PEP 006 outputs api #2763
PEP 006 outputs api #2763
Conversation
We talked about putting this behind a feature flag, that's probably something we should do initially. |
author Brian DeGeeter <[email protected]> 1673381173 -0800 committer Brian DeGeeter <[email protected]> 1684277785 -0700 parent 8ff74f4 author Brian DeGeeter <[email protected]> 1673381173 -0800 committer Brian DeGeeter <[email protected]> 1684277652 -0700 parent 8ff74f4 author Brian DeGeeter <[email protected]> 1673381173 -0800 committer Brian DeGeeter <[email protected]> 1684277541 -0700 chore: basic structure for grpc api Signed-off-by: Brian DeGeeter <[email protected]> chore: move make targets to mage Signed-off-by: Brian DeGeeter <[email protected]> wip: buf scaffolding Signed-off-by: Steven Gettys <[email protected]> chore: Added simple test client/server chore: move grpc test under tests folder Signed-off-by: Brian DeGeeter <[email protected]> chore: add basic grpc server test Signed-off-by: Brian DeGeeter <[email protected]> chore: retire makefile Signed-off-by: Brian DeGeeter <[email protected]> chore: collab on failing porter integration Signed-off-by: Brian DeGeeter <[email protected]> Feat/add server to porter cmd (#2) * wip: Saving state * wip: Added per rpc connection * chore: Removed debug prints chore: Added server cli options Signed-off-by: Steven Gettys <[email protected]> chore: Fixed int viper conversion error added grpc context for porter connection feat: update proto file chore: Generated new go grpc and moved test over to using the porter connection in the grpc context chore: Added UT for grpc context package Signed-off-by: Steven Gettys <[email protected]> fill out installation list grpc response use PorterTest for grpc integration tests chore: Updated grpc integration tests to inject installations into the porter test client Signed-off-by: Steven Gettys <[email protected]> porter structs to grpc via json marshaling chore: convert store install to display for grpc integ test chore: Added UT for grpc context and installation packages Signed-off-by: Steven Gettys <[email protected]> chore: Consolidated duplicate test code for grpc installation service chore: Refactored to use a single portergrpc package' Signed-off-by: Steven Gettys <[email protected]> chore: Renamed grpc PorterBundleServer to PorterServer Signed-off-by: Steven Gettys <[email protected]> chore: Checked in proto change Signed-off-by: Steven Gettys <[email protected]> chore: MOved around grpc package some Signed-off-by: Steven Gettys <[email protected]> chore: refactor grpc integ test chore: remove porter-service cmd files chore: refactor for integ testing chore: add cred and param sets to installation proto intial run outputs grpc func refactor outputs proto to remove extra depth chore: add test intallation opts for data driven grpc integ testing chore: Testing out action (#3) * chore: Testing out action Signed-off-by: Steven Gettys <[email protected]> * chore: Removed the DOCKER_BUILDKIT flag Signed-off-by: Steven Gettys <[email protected]> * chore: Removed docker buildkit Signed-off-by: Steven Gettys <[email protected]> * chore: Added default case Signed-off-by: Steven Gettys <[email protected]> --------- Signed-off-by: Steven Gettys <[email protected]> Bump github.com/moby/buildkit from 0.10.6 to 0.11.0 (getporter#2520) * Bump github.com/moby/buildkit from 0.10.6 to 0.11.0 Signed-off-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Yingrong Zhao <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Yingrong Zhao <[email protected]> Bump github.com/google/go-containerregistry from 0.12.1 to 0.13.0 Bumps [github.com/google/go-containerregistry](https://github.com/google/go-containerregistry) from 0.12.1 to 0.13.0. - [Release notes](https://github.com/google/go-containerregistry/releases) - [Changelog](https://github.com/google/go-containerregistry/blob/main/.goreleaser.yml) - [Commits](google/go-containerregistry@v0.12.1...v0.13.0) --- updated-dependencies: - dependency-name: github.com/google/go-containerregistry dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Address go-restful CVE-2022-1996 (getporter#2547) * Address go-restful CVE-2022-1996 * Upgrade go-restful to v3 which addresses CVE-2022-1996 * Upgrade containerd to v1.6.16 * Upgrade cnab-go to v1.25.0 * Upgrade docker to v23.0.0-rc.1 * Upgrade buildx to v0.10.2 which required some careful code changes to compile again Signed-off-by: Carolyn Van Slyck <[email protected]> * Pin cnab-go to v0.25.0 Signed-off-by: Carolyn Van Slyck <[email protected]> --------- Signed-off-by: Carolyn Van Slyck <[email protected]> Bump github.com/stretchr/testify from 1.8.1 to 1.8.2 Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.1 to 1.8.2. - [Release notes](https://github.com/stretchr/testify/releases) - [Commits](stretchr/testify@v1.8.1...v1.8.2) --- updated-dependencies: - dependency-name: github.com/stretchr/testify dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Bump github.com/grpc-ecosystem/go-grpc-middleware from 1.3.0 to 1.4.0 Bumps [github.com/grpc-ecosystem/go-grpc-middleware](https://github.com/grpc-ecosystem/go-grpc-middleware) from 1.3.0 to 1.4.0. - [Release notes](https://github.com/grpc-ecosystem/go-grpc-middleware/releases) - [Commits](grpc-ecosystem/go-grpc-middleware@v1.3.0...v1.4.0) --- updated-dependencies: - dependency-name: github.com/grpc-ecosystem/go-grpc-middleware dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Bump github.com/spf13/afero from 1.9.4 to 1.9.5 Bumps [github.com/spf13/afero](https://github.com/spf13/afero) from 1.9.4 to 1.9.5. - [Release notes](https://github.com/spf13/afero/releases) - [Commits](spf13/afero@v1.9.4...v1.9.5) --- updated-dependencies: - dependency-name: github.com/spf13/afero dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Bump github.com/hashicorp/go-hclog from 1.4.0 to 1.5.0 Bumps [github.com/hashicorp/go-hclog](https://github.com/hashicorp/go-hclog) from 1.4.0 to 1.5.0. - [Release notes](https://github.com/hashicorp/go-hclog/releases) - [Commits](hashicorp/go-hclog@v1.4.0...v1.5.0) --- updated-dependencies: - dependency-name: github.com/hashicorp/go-hclog dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Bump github.com/google/go-containerregistry from 0.13.0 to 0.14.0 Bumps [github.com/google/go-containerregistry](https://github.com/google/go-containerregistry) from 0.13.0 to 0.14.0. - [Release notes](https://github.com/google/go-containerregistry/releases) - [Changelog](https://github.com/google/go-containerregistry/blob/main/.goreleaser.yml) - [Commits](google/go-containerregistry@v0.13.0...v0.14.0) --- updated-dependencies: - dependency-name: github.com/google/go-containerregistry dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Bump github.com/containerd/containerd from 1.6.19 to 1.7.0 Bumps [github.com/containerd/containerd](https://github.com/containerd/containerd) from 1.6.19 to 1.7.0. - [Release notes](https://github.com/containerd/containerd/releases) - [Changelog](https://github.com/containerd/containerd/blob/main/RELEASES.md) - [Commits](containerd/containerd@v1.6.19...v1.7.0) --- updated-dependencies: - dependency-name: github.com/containerd/containerd dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Revert "Merge pull request getporter#2653 from getporter/dependabot/go_modules/github.com/google/go-containerregistry-0.14.0" This reverts commit 7ee355f, reversing changes made to 09297d6. Signed-off-by: Carolyn Van Slyck <[email protected]> Bump go.mongodb.org/mongo-driver from 1.11.3 to 1.11.4 Bumps [go.mongodb.org/mongo-driver](https://github.com/mongodb/mongo-go-driver) from 1.11.3 to 1.11.4. - [Release notes](https://github.com/mongodb/mongo-go-driver/releases) - [Commits](mongodb/mongo-go-driver@v1.11.3...v1.11.4) --- updated-dependencies: - dependency-name: go.mongodb.org/mongo-driver dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> chore: basic structure for grpc api Signed-off-by: Brian DeGeeter <[email protected]> Signed-off-by: Steven Gettys <[email protected]> Co-authored-by: Brian DeGeeter <[email protected]> Co-authored-by: Steven Gettys <[email protected]> chore: fix tests after rebase Signed-off-by: Brian DeGeeter <[email protected]> chore: tidy up go.sum Signed-off-by: Brian DeGeeter <[email protected]> chore: remove wip file Signed-off-by: Brian DeGeeter <[email protected]>
…ifest change (google/go-containerregistry#1551) Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Brian DeGeeter <[email protected]>
Bumps [github.com/docker/cli](https://github.com/docker/cli) from 23.0.3+incompatible to 23.0.6+incompatible. - [Commits](docker/cli@v23.0.3...v23.0.6) --- updated-dependencies: - dependency-name: github.com/docker/cli dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Bumps [go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp](https://github.com/open-telemetry/opentelemetry-go) from 1.14.0 to 1.15.1. - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.14.0...v1.15.1) --- updated-dependencies: - dependency-name: go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: schristoff <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Bumps [github.com/docker/distribution](https://github.com/docker/distribution) from 2.8.1+incompatible to 2.8.2+incompatible. - [Release notes](https://github.com/docker/distribution/releases) - [Commits](distribution/distribution@v2.8.1...v2.8.2) --- updated-dependencies: - dependency-name: github.com/docker/distribution dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Tomi Paananen <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Tomi Paananen <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Tomi Paananen <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Tomi Paananen <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Tomi Paananen <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Tomi Paananen <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Tomi Paananen <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Tomi Paananen <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Tomi Paananen <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Tomi Paananen <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Tomi Paananen <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Arhell <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 23.0.3+incompatible to 23.0.6+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](moby/moby@v23.0.3...v23.0.6) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Troy Connor <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Troy Connor <[email protected]> Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Brian DeGeeter <[email protected]> chore: move make targets to mage Signed-off-by: Brian DeGeeter <[email protected]> wip: buf scaffolding Signed-off-by: Steven Gettys <[email protected]> chore: Added simple test client/server chore: move grpc test under tests folder Signed-off-by: Brian DeGeeter <[email protected]> chore: add basic grpc server test Signed-off-by: Brian DeGeeter <[email protected]> chore: retire makefile Signed-off-by: Brian DeGeeter <[email protected]> chore: collab on failing porter integration Signed-off-by: Brian DeGeeter <[email protected]> Feat/add server to porter cmd (#2) * wip: Saving state * wip: Added per rpc connection * chore: Removed debug prints chore: Added server cli options Signed-off-by: Steven Gettys <[email protected]> chore: Fixed int viper conversion error added grpc context for porter connection feat: update proto file chore: Generated new go grpc and moved test over to using the porter connection in the grpc context chore: Added UT for grpc context package Signed-off-by: Steven Gettys <[email protected]> fill out installation list grpc response use PorterTest for grpc integration tests chore: Updated grpc integration tests to inject installations into the porter test client Signed-off-by: Steven Gettys <[email protected]> porter structs to grpc via json marshaling chore: convert store install to display for grpc integ test chore: Added UT for grpc context and installation packages Signed-off-by: Steven Gettys <[email protected]> chore: Consolidated duplicate test code for grpc installation service chore: Refactored to use a single portergrpc package' Signed-off-by: Steven Gettys <[email protected]> chore: Renamed grpc PorterBundleServer to PorterServer Signed-off-by: Steven Gettys <[email protected]> chore: Checked in proto change Signed-off-by: Steven Gettys <[email protected]> chore: MOved around grpc package some Signed-off-by: Steven Gettys <[email protected]> chore: refactor grpc integ test chore: remove porter-service cmd files chore: refactor for integ testing chore: add cred and param sets to installation proto intial run outputs grpc func refactor outputs proto to remove extra depth chore: add test intallation opts for data driven grpc integ testing chore: Testing out action (#3) * chore: Testing out action Signed-off-by: Steven Gettys <[email protected]> * chore: Removed the DOCKER_BUILDKIT flag Signed-off-by: Steven Gettys <[email protected]> * chore: Removed docker buildkit Signed-off-by: Steven Gettys <[email protected]> * chore: Added default case Signed-off-by: Steven Gettys <[email protected]> --------- Signed-off-by: Steven Gettys <[email protected]> chore: address PR feedback Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Brian DeGeeter <[email protected]>
3e00da1
to
c5834b1
Compare
Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Brian DeGeeter <[email protected]>
Signed-off-by: Steven Gettys <[email protected]>
Signed-off-by: Brian DeGeeter <[email protected]>
This command starts the gRPC server for porter which is able to expose limited porter functionality via RPC. | ||
Currently only data operations are supported, creation of resources such as installations, credential sets, or parameter sets is not supported. | ||
|
||
A list of the supported RPCs can be found at <link?> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
porter -> Porter (it is the nittiest of nits, but I only included it because I'm going to point out....) -> and <link?> ?
// protoc https://github.com/protocolbuffers/protobuf/releases/download/v21.12/protoc-21.12-linux-x86_64.zip`` | ||
//TODO: add more tools | ||
// https://github.com/bufbuild/buf/releases protoc-gen-buf-breaking, protoc-gen-buf-lint | ||
// protoc https://github.com/protocolbuffers/protobuf/releases/download/v21.12/protoc-21.12-linux-x86_64.zip`` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this the same TODO twice or am I not seeing the difference? If it's different then it's my b
@@ -17,3 +29,67 @@ func EnsureProtobufTools() { | |||
VersionCommand: "--version", | |||
})) | |||
} | |||
|
|||
// IsCommandInPath determines if a command can be called based on the current PATH. | |||
func IsCommandInPath(cmd string) (bool, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we return one of the two here?
If we return error, then I expect for what we're doing here to be something with severity (as in, maybe we should stop if we get an error)
But if it's something we can keep running through, then just returning the bool and handling it in the caller function is probably fine
|
||
func EnsureBufBuild() { | ||
mg.Deps(EnsureProtobufTools) | ||
if ok, _ := pkg.IsCommandAvailable("buf", "--version", "1.11.0"); ok { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Silly question - I may be misreading, feel free to correct. Are we sticking ourselves to a specific version of this? Is there anyway we could tie ourselves to 'latest' if we are, to ensure we don't have to come update this later, or is there a reason we want this specific version?
import ( | ||
"context" | ||
|
||
pGRPC "get.porter.sh/porter/gen/proto/go/porterapis/porter/v1alpha1" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pGRPC -> v1alpha1 (to stay consistent with everywhere else?)
|
||
// Create a customized counter metric. | ||
customizedCounterMetric = prometheus.NewCounterVec(prometheus.CounterOpts{ | ||
Name: "demo_server_say_hello_method_handle_count", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we update this name or does it not matter?
return srv, nil | ||
} | ||
|
||
func (s *PorterGRPCService) ListenAndServe() (*grpc.Server, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some comments here would be majorly helpful!
"strconv" | ||
"testing" | ||
|
||
iGRPC "get.porter.sh/porter/gen/proto/go/porterapis/installation/v1alpha1" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same comment about versioning here
for i, gPV := range actual.GetOutputs() { | ||
bActOut, err := pjm.Marshal(gPV) | ||
require.NoError(t, err) | ||
//TODO: make this not dependant on order |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like assert.JSONEq
isn't dependent on order, is that what this TODO is referencing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Leaving my notes for future selves
What does this change
This change adds a hidden argument that enables a GRPC service. This new service is the start of PEP 006 and adds a API allowing access to installation output information. This feature is only intended to be used by the operator and will require additional functional to leverage.
https://github.com/getporter/proposals/blob/main/pep/006-installation-outputs-api.md
This feature is not intended to be used directly via the porter CLI by the end user. An image is created containing a version of the porter cli with a build flag to enable the
api-server
command. This image is intended to be used by a future kubernetes deployment with the Porter Operator. The regular porter cli build is not built withapi-server
command.Notes for the reviewer
An update to go-containerregistry required a change to existing archive integration tests. The change to this library introduces a new field to the index.json in an archive. This required the shasum for the test to be updated.
Documentation will come in as part of additional operator integration work.
Checklist
Reviewer Checklist