Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate away from the kubernetes-release bucket #3729

Open
puerco opened this issue Aug 27, 2024 · 9 comments
Open

Migrate away from the kubernetes-release bucket #3729

puerco opened this issue Aug 27, 2024 · 9 comments
Labels
kind/feature Categorizes issue or PR as related to a new feature. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.

Comments

@puerco
Copy link
Member

puerco commented Aug 27, 2024

Context

Release artifact have historically been published to a GCS bucket called kubernetes-release, this bucket is not under community control. As of the week of Aug 19th the release artifacts are being served from a CDN backed by a community bucket.

Currently, the contents of kubernetes-release are mirrored to the community bucket for serving every two hours.

/cc @kubernetes/release-engineering

TODO

Identify processes that need to be moved

We need to comprehensively search our processes to find those relying on data from kubernetes-release. Once we have an idea of which ones are reading and/or writing to the google owned bucket, let's expand the lists below with those that need to be migrated.

Migrating to the community bucket involves two groups of tasks, let's expand these as we find them:

Kubernetes Release Process

  • TBD

Internal Processes and Tests

@k8s-ci-robot k8s-ci-robot added needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-priority labels Aug 27, 2024
@ameukam
Copy link
Member

ameukam commented Aug 28, 2024

We had many conversations about this in kubernetes/k8s.io#2396.

@BenTheElder
Copy link
Member

We should also confirm the GCB project being used.

We need to comprehensively search our processes to find those relying on data from kubernetes-release. Once we have an idea of which ones are reading and/or writing to the google owned bucket, let's expand the lists below with those that need to be migrated.

I believe expected writes are only krel? There's a constant in krel for the bucket.

For reads, we've already made a big push to point things at dl.k8s.io instead, if any more crop up we can fix them later as worst case they won't have new releases until they switch, and the new bucket is intentionally not public read (only through the CDN).

I think it should be:

  • make sure krel GCB service account has write to the new bucket (should be done already but double check)
  • swap krel to write to the new bucket
  • spin down sync job
  • confirm next release goes smoothly
  • googler (me): drop remaining write permissions to legacy google-containers project

@ameukam
Copy link
Member

ameukam commented Sep 3, 2024

make sure krel GCB service account has write to the new bucket (should be done already but double check)

It's not done yet. krel leverage the GCB service agent of the kubernetes-release-test GCP project to cut releases.

@BenTheElder
Copy link
Member

We should also migrate out of kubernetes-release-test which is in google.com to a project in kubernetes.io, but we could do that in two phases.

@ameukam
Copy link
Member

ameukam commented Sep 3, 2024

We should also migrate out of kubernetes-release-test which is in google.com to a project in kubernetes.io, but we could do that in two phases.

See: #3425

@ameukam
Copy link
Member

ameukam commented Sep 9, 2024

/kind feature
/priority important-soon

@k8s-ci-robot k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-priority labels Sep 9, 2024
ameukam added a commit to ameukam/release that referenced this issue Sep 9, 2024
Part of:
 - kubernetes#3729

Use a community-bucket for Kubernetes releases
ameukam added a commit to ameukam/k8s.io that referenced this issue Sep 9, 2024
Ref:
  - kubernetes/release#3729

Temporary allow the Service Agent for the GCB Service from project
`kubernetes-release-test`. This will enable artifacts release for
Kubernetes to a community-owned bucket.
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 8, 2024
@mengjiao-liu
Copy link
Member

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 9, 2024
@BenTheElder
Copy link
Member

I think this is all but done, the bucket isn't being written to anymore but I should drop permissions to it on the Google side and we still? need to move the krel jobs out of the internal project into a community project. Not sure if we had a separate tracking issue for the latter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Projects
None yet
Development

No branches or pull requests

6 participants