Helm Chart resource should allow to skip CRD installation

[aylei]

Problem description

• I saw an issue in the following Pulumi Program: https://github.com/pulumi/pulumi-kubernetes

Errors & Logs

Example project:

import * as k8s from "@pulumi/kubernetes";

const albController = new k8s.helm.v3.Chart("aws-load-balancer-controller", {
  namespace: "kube-system",
  chart: "aws-load-balancer-controller",
  version: "1.1.0",
  values: {
    "clusterName": "test",
  },
  fetchOpts: {
    repo: "https://aws.github.io/eks-charts",
  },
});

Preview results:

Previewing update (test):
     Type                                                                          Name
 +   pulumi:pulumi:Stack                                                           test-test
 +   └─ kubernetes:helm.sh:Chart                                                   aws-load-balancer-controller
 +      ├─ kubernetes:rbac.authorization.k8s.io:ClusterRoleBinding                 aws-load-balancer-controller-rolebinding
 +      ├─ kubernetes:admissionregistration.k8s.io:ValidatingWebhookConfiguration  aws-load-balancer-webhook
 +      ├─ kubernetes:rbac.authorization.k8s.io:ClusterRole                        aws-load-balancer-controller-role
 +      ├─ kubernetes:core:Service                                                 kube-system/aws-load-balancer-webhook-service
 +      ├─ kubernetes:rbac.authorization.k8s.io:RoleBinding                        kube-system/aws-load-balancer-controller-leader-election-rolebin
 +      ├─ kubernetes:admissionregistration.k8s.io:MutatingWebhookConfiguration    aws-load-balancer-webhook
 +      ├─ kubernetes:rbac.authorization.k8s.io:Role                               kube-system/aws-load-balancer-controller-leader-election-role
 +      ├─ kubernetes:core:Secret                                                  kube-system/aws-load-balancer-tls
 +      ├─ kubernetes:apps:Deployment                                              kube-system/aws-load-balancer-controller
 +      └─ kubernetes:apiextensions.k8s.io:CustomResourceDefinition                targetgroupbindings.elbv2.k8s.aws

Note that the preview results include kubernetes:apiextensions.k8s.io:CustomResourceDefinition targetgroupbindings.elbv2.k8s.aws, which is not included in the helm template output of the helm cli with same version:

> helm version
version.BuildInfo{Version:"v3.4.1", GitCommit:"c4e74854886b2efe3321e185578e6db9be0a6e29", GitTreeState:"clean", GoVersion:"go1.14.11"}
> helm template test eks/aws-load-balancer-controller --version=1.1.0 -f values.yaml | grep Custom
(no customresourcedefinition found)

The CustomResourceDefinition is actually included in the chart project, but not in the ./templates directory: https://github.com/aws/eks-charts/tree/master/stable/aws-load-balancer-controller which should have not been rendered by helm.

Affected product version(s)

pulumi-kubernetes v2.7.3

Note: I was using pulumi-kubernetes v1.6.0, and not seeing this issue.

Reproducing the issue

As described above.

Suggestions for a fix

[Comradin]

These CRDs are from the crd subdirectory and belong to kustomize and in the case of this particular chart are quite "dangerous" as they contain old apiVersions which EKS will drop with the 1.19+ Releases.

[Comradin]

Any news from this issue? These extra resources are quite annoying

[nesl247]

This is due to pulumi running helm template --include-crds. I don't recall if you can disable the inclusion of CRDs in Pulumi. However, helm install with include the CRDs by default from https://helm.sh/docs/chart_best_practices/custom_resource_definitions/, which means that the helm chart you linked to is probably what is wrong.

However, I'm not very familiar with kustomize, so I don't know if maybe helm does something to ignore its files.

[EronWright]

It is true that Pulumi automatically installs the CRDs that are included with the chart (ie helm template --include-crds). Let's consider this ticket to be a feature request for a skipCrds option on Chart as seen on Release.

[EronWright]

The upcoming Chart v4 resource now supports a skipCrds input property, to skip installation of CRDs in the crds/ special directory within the chart.

Like with Release resource, the skipCrds property doesn't skip over CRDs that are installed via templates; use a cahrt-specific value like installCrds: false to accomplish that.