profiles

Gitops native package management.

• Getting started
  • Dependencies
  • Local environment using Kind
  • Installing Profiles
• Development
  • Tests
  • Release process
  • Dev Tags
  • Generating the protocol files
• Terminology
  • Profile
  • Catalog
  • Profile Catalog Source
  • Profile Catalog Source Controller
• Current Architecture
  • Catalogs and Sources
  • Profile Installation
• Roadmap
  • Profiles
  • Catalogs

Getting started

Dependencies

The dependency libgit2 v1.1.0 needs to be installed to be able to run the Controller or its test-suite locally. Docs on install libgit2

buf is needed to generate and lint protoc code. For more details check out Generating the protocol files.

Local environment using Kind

1. Set up local environment: make local-env.

   This will start a local kind cluster and installs the profiles and flux components.

2. Deploy an example catalog source kubectl apply -f examples/profile-catalog-source.yaml

Installing Profiles

1. Profiles can be installed using pctl.

Development

Tests

1. All tests can be run with make test.

2. Acceptance tests can be run with make acceptance.

3. For further commands, run make help.

Release process

There are some manual steps right now, should be streamlined soon.

Steps:

1. Create a new release notes file:

   touch docs/release_notes/<version>.md

2. Copy-and paste the release notes from the draft on the releases page into this file. Note: sometimes the release drafter is a bit of a pain, verify that the notes are correct by doing something like: git log --first-parent tag1..tag2.

3. PR the release notes into main.

4. Create and push a tag with the new version:

   git tag <version>
   git push origin <version>

5. The Create release action should run. Verify that:

   1. The release has been created in Github
      1. With the correct assets
      2. With the correct release notes
   2. The image has been pushed to docker
   3. The image can be pulled and used in a deployment

Dev Tags

As part of pushing a new branch to profiles, a new dev tag will be created for that branch. This action is part of the procedure around working with profiles and pctl, its companion CLI tool. The details of this are explained in pctl's README section Working with Profiles.

The dev tag's format is as follows: <currentLatestReleaseTag>-<branch-name>.

Generating the protocol files

profiles is using buf to generate and lint protobuf implementations. The protocol descriptor file can be found here profiles.proto. To update these files, install buf and run the make target make generate-protoc. Profiles, using the tools way to track dependencies, and should already have all the necessary protoc tools installed.

When running buf for the first time, install all the necessary libraries by running buf beta mod update. This will install dependencies defined in buf.yaml file.

What gets generated is defined by buf.gen.yaml file. For the document target to work, install protoc-gen-doc by running go get -u github.com/pseudomuto/protoc-gen-doc/cmd/protoc-gen-doc.

Documentation about the protoc files can be found under docs/index.html.

Terminology

Profile

A Profile is a "package" of Kubernetes deployable objects, known as Artifacts, and configurable values. Artifacts are one of: Helm Chart; Helm Release; raw yaml; Kustomize patch; Profile (nested).

For an example, see the profiles-examples.

Catalog

A Catalog is an in-memory cache of Profiles. There is one Catalog per running Profile Controller. The Catalog is queryable via pctl or the API directly which runs alongside the Profiles Controller.

Profiles can be added to the Catalog by creating a ProfileCatalogSource.

Profile Catalog Source

A ProfileCatalogSource is a custom resource through which approved Profiles can be managed in the Catalog

// ProfileCatalogSourceSpec defines the desired state of ProfileCatalogSource
type ProfileCatalogSourceSpec struct {
	// Profiles is the list of profiles exposed by the catalog
	Profiles []ProfileDescription `json:"profiles,omitempty"`
}

// ProfileDescription defines details about a given profile.
type ProfileDescription struct {
	// Profile name
	Name string `json:"name,omitempty"`
	// Profile description
	Description string `json:"description,omitempty"`
	// Version
	// +optional
	Version string `json:"version,omitempty"`
	// CatalogSource is the name of the catalog the profile is listed in
	// +optional
	CatalogSource string `json:"catalog,omitempty"`
	// URL is the full URL path to the profile.yaml
	// +optional
	URL string `json:"url,omitempty"`
	// Maintainer is the name of the author(s)
	// +optional
	Maintainer string `json:"maintainer,omitempty"`
	// Prerequisites are a list of dependencies required by the profile
	// +optional
	Prerequisites []string `json:"prerequisites,omitempty"`
}

Profiles can therefore be grouped and namespaced within the Catalog.

Profile Catalog Source Controller

The Profile Catalog Source Controller reconciles ProfileCatalogSource resources. See architecture diagrams below for what the reconciliation process does.

Current Architecture

Catalogs and Sources

Illustration of how Profiles are added to the Catalog and how they can then be queried via the Catalog API:

Profile Installation

To see how a profile installation works, take a look at the pctl documentation

Roadmap

Profiles

Install:

• Install a simple profile which contains a single Helm release artifact
• Install a simple profile which contains a raw yaml artifact (k8s object manifest)
• Install a simple profile which contains another profile (single nesting)
• Install a profile which contains a mix of all artifact types
• Install a profile which contains nested profiles to depth N
• Install a profile with pctl in a gitops way (ie there is a PR involved, and nobody touches the cluster)
• Install a profile which is listed in the catalog
• Install a profile which is NOT listed in the catalog
• Install a private profile

Configure:

• Configure a Helm release artifact installation
• Apply Kustomise patches
• Configure different values across multiple artifacts

Update:

• Discover when there is a newer version available
• Update a profile

Catalogs

Catalog sources:

• Create a catalog source
• Delete a catalog source
• Grant/Revoke access to CatalogSources

Catalog management:

• Install profiles to the catalog
• Update profiles in the catalog
• Delete profiles from the catalog

API:

• Search for profiles in the catalog
• Get more information about a profile in the catalog