Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resources: support remote files #970

Closed
pohly opened this issue Apr 12, 2019 · 23 comments · Fixed by #2167
Closed

resources: support remote files #970

pohly opened this issue Apr 12, 2019 · 23 comments · Fixed by #2167

Comments

@pohly
Copy link
Contributor

pohly commented Apr 12, 2019

Currently, resources entries must be local files. It would be useful to allow references to remote files, for example a file served by a HTTPS server.

The use case I have in mind is about pulling the rbac.yaml files that Kubernetes-CSI publishes for each sidecar app into a deployment:
https://github.com/kubernetes-csi/external-provisioner/blob/master/deploy/kubernetes/rbac.yaml

Originally this was suggested in #254, but that got closed by just updating the docs. #923 is related, but covers only bases.

Pulling remote files has security implications. When using HTTPS, one has to trust the remote server and relies on HTTPS security to prevent man-in-the-middle attacks. It would be nice to support defining and checking an expected file content hash.

@Liujingfang1
Copy link
Contributor

resources should be from local disks, which is different from the bases. We don't plan to support remote files for resources.
What you could do is to submit a PR to kubernetes-csi/external-provisioner, adding a kustomization.yaml there.
Then you can use it as a remote base.

@pohly
Copy link
Contributor Author

pohly commented May 16, 2019 via email

@alexpkalinka
Copy link

alexpkalinka commented May 31, 2019

resources should be from local disks

Can you explain why? What is the problem with loading remote resources? If a base kustomization.yaml is loaded from a remote server then it eventually loads resources from a remote server too, right? Then why can't we just load remote resources directly, bypassing loading a base?

@benjamin-bergia
Copy link

Forcing people to have this kustomization.yml on their repo makes the remote base feature pretty much useless in real life scenarios. The idea of being able to load some yaml from whatever repo on github, untouched, and create variants from it will not happen if we expect repo maintainers to create kustomizations. In an other hand, if we allow remote resources we automatically give access to any resources to all kustomize users. A work around would be to use git submodules and a kustomization.yml file but that's ugly.

@evrardjp
Copy link

I would like to consume resources directly from URL.

I don't see a security issue if the manifest from the resource is audited before use, and I don't see a good technical reason why it cannot be achieved. Could you clarify why there is no plan for this?

I even don't mind tackling the code on my spare time, but I need a longer explanation than We don't plan to support remote files for resources.

@longility
Copy link

This is my need for kustomize to support remote files.

I would like to manage my k8s resources in a consistent and declarative way as much as possible. I'm trying out to not be dependent on helm. Kustomize is definitely working out for me at the moment. However, being able to reference remote file urls will help with referencing 3rd party yamls such as for cert-manager, ingress-nginx, jaeger, and kubernetes dashboard. At this time, I'm pulling the code into my repository.

Is there a cleaner way that I don't know about?

@paulgmiller
Copy link

Has anyone brought this up at the kustomize sig meeting. Seems like it would be trivial to make a pr to push the discussion further but I agree it's not really clear why from @Liujingfang1 remote resources couln't be done.

@paulgmiller
Copy link

Reflected this over to kubernetes/kubernetes#82459 in case kustomize development has moved there.

@paulgmiller
Copy link

Another fun example would be installing knative https://github.com/knative/serving/releases/tag/v0.8.1 Maybe should provide a kustomization.yaml but as is they provide a couple of yamls you can combine manually love to refernce them directly from my kustomziation.yaml rather than mirroring.

@sbose78
Copy link

sbose78 commented Sep 10, 2019

I see an example with a remote resources file

DEMO_HOME=$(mktemp -d)

cat <<EOF >$DEMO_HOME/kustomization.yaml
resources:
- github.com/kubernetes-sigs/kustomize//examples/multibases?ref=v1.0.6
namePrefix: remote-
EOF

https://github.com/kubernetes-sigs/kustomize/blob/master/examples/remoteBuild.md#remote-targets

It doesn't work.

@Liujingfang1
Copy link
Contributor

@sbose78 Which version do you use? I checked with HEAD and it works.

@evrardjp
Copy link

evrardjp commented Sep 11, 2019

@Liujingfang1 does that work for resources that aren't a base? Resources which are bases are fine, it's for the rest of the resources that remote data could be useful.

@lukasmrtvy
Copy link

lukasmrtvy commented Dec 3, 2019

@Liujingfang1 does not work for me, ( kubectl v1.16.3, it should use 2.0.3 -> https://github.com/kubernetes/kubernetes/blob/release-1.16/go.mod#L487)

DEMO_HOME=$(mktemp -d)

cat <<EOF >$DEMO_HOME/kustomization.yaml
resources:
- github.com/kubernetes-sigs/kustomize//examples/multibases?ref=v1.0.6
namePrefix: remote-
EOF

kubectl kustomize .

Error: rawResources failed to read Resources: Load from path github.com/kubernetes-sigs/kustomize//examples/multibases?ref=v1.0.6 failed: evalsymlink failure on '/tmp/tmp.AotZwipVEC/github.com/kubernetes-sigs/kustomize/examples/multibases?ref=v1.0.6' : lstat /tmp/tmp.AotZwipVEC/github.com: no such file or directory

I tried standalone kustomize binary 3.4.0 and its working also with resources.
The problem is that You have to need kustomize.yaml inside remote repository, which is useless ( in real world examples ) when You want to include manifest from 3rd party repository that does not use kustomize.

@kYann
Copy link

kYann commented Dec 18, 2019

Same here, I need this feature, and needing the kustomize.yaml on the remote side is not possible.

@JohnPaton
Copy link

JohnPaton commented Jan 2, 2020

@Liujingfang1 Also putting in my support for this feature. I've had PRs on a couple of projects now trying to get maintainers to add kustomization.yamls which, while good for kustomize's adoption rate, is obviously cumbersome.

Latest example: Here Google provides a DaemonSet for installing Nvidia drivers on nodes which I would like to have included in my build, but it's not possible because I can't use their file directly and they don't have a kustomization.yaml (yet).

The first reply to this issue that

resources should be from local disks, which is different from the bases

also doesn't really apply anymore since bases has been deprecated since 2.1.0; resources is the only place left

@cmorent
Copy link

cmorent commented Jan 20, 2020

Any update on this issue? I also need this for quite a simple use case: I want to install the Jaeger operator on one of my clusters, so far they only provide yaml files to run the required resources but no kustomize related config so I can't import the base. I only want to have something like the following in my own config:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: default

resources:
- https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/crds/jaegertracing.io_jaegers_crd.yaml

For now, if I understand well, the best solution I have is to copy the remote file content into a local file.

@alwaysastudent
Copy link

Is this supported now?

@DiegoPomares
Copy link

DiegoPomares commented Aug 31, 2021

For anyone stumbling upon this issue while googling:

It is supported now, but the documentation is outdated as of today as this URL specifications should follow the hashicorp URL format. The directory must contain a kustomization.yaml file. is not true, at least for the standalone kustomize v4.3.0.

Just put a regular URL in the resources section, example:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

@karpoftea
Copy link

It seems that there is not way to use git-protocol for referencing individual files. So, following will fail:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - [email protected]:argoproj/argo-cd.git/manifests/crds/application-crd.yaml?ref=v2.1.3

with error:

no such file or directory': '/private/var/folders/4x/3kz53w8j2jvgsrzdctn966v40000gn/T/kustomize-194475774/manifests/crds/application-crd.yaml' refers to file 'application-crd.yaml'; expecting directory

While following will work (mind that crds dir has kustimization.yaml inside):

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - [email protected]:argoproj/argo-cd.git/manifests/crds?ref=v2.1.3

In my use-case git-repo is private, thus I can't use 'https'-protocol for fetching individual files.

Why there is a limitation for individual files for git-protocol?
(upvote please if this feature deserves a separate issue)

@btrepp
Copy link

btrepp commented Apr 14, 2022

This is a bit of a weird one to stumble upon. Getting the file from git would be a great advantage.
Some tools have multiple versions of the .yaml in the directory. E.g argocd. So if you kustomize the whole folder, you will have a bad time.

With the git protocol, we could use the commit hash, so you would also be confident someone sneaky isn't changing the files, and have some form of verification. In the https mode, that's just a URL, someone could change that file and you probably wouldn't know, or get any indication you just built something malicious

@tonglil
Copy link

tonglil commented Apr 7, 2023

Two years later and running into the issue @karpoftea described above.

Really unknown why the limitation is imposed on the git loader. The https loader is useless in an enterprise context, where storing kustomize files elsewhere is needed...

@HWiese1980
Copy link

Has there been any progress here?

@joebowbeer
Copy link
Contributor

The original issue was fixed years ago as this example illustrates:

Resources can reference remote files via their raw GitHub urls, such as
- https://raw.githubusercontent.com/kubernetes-sigs/kustomize/8ea501347443c7760217f2c1817c5c60934cf6a5/examples/helloWorld/deployment.yaml

References to remote directories also works.

For references to remote files in private repos using schemes other than file or https, I suggest creating a new issue.

However, given kustomize's preference for folder refs and its restrictions on loading resources not in or below the root folder, even the support for remote files already provided seems questionable to me.

https://kubectl.docs.kubernetes.io/faq/kustomize/#security-file-foo-is-not-in-or-below-bar

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.