Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make Kubernetes aware of the LoadBalancer behaviour #1860

Open
12 tasks done
Sh4d1 opened this issue Jun 13, 2020 · 72 comments
Open
12 tasks done

Make Kubernetes aware of the LoadBalancer behaviour #1860

Sh4d1 opened this issue Jun 13, 2020 · 72 comments
Assignees
@thockin @rikatz @RyanAoh
Labels
lead-opted-in Denotes that an issue has been opted in to a release sig/network Categorizes an issue or PR as relevant to SIG Network. stage/beta Denotes an issue tracking an enhancement targeted for Beta status
Milestone
v1.32

Comments

@Sh4d1
Copy link
Member

Sh4d1 commented Jun 13, 2020 •
edited by pacoxu
Loading

Enhancement Description

Please to keep this description up to date. This will help the Enhancement Team track efficiently the evolution of the enhancement
@k8s-ci-robot k8s-ci-robot added the needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label Jun 13, 2020
@Sh4d1
Copy link
Member Author

Sh4d1 commented Jun 13, 2020

/sig network

@k8s-ci-robot k8s-ci-robot added sig/network Categorizes an issue or PR as relevant to SIG Network. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jun 13, 2020
@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 11, 2020
@Sh4d1
Copy link
Member Author

Sh4d1 commented Sep 11, 2020

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 11, 2020
@kikisdeliveryservice
Copy link
Member

Hi @Sh4d1

1.20 Enhancements Lead here. This KEP still seems to be a draft (it's provisional) but wanted to check-in to see if you thought this would be graduating alpha in 1.20?

Enhancements Freeze is October 6th and by that time we require:

The KEP must be merged in an implementable state
The KEP must have test plans
The KEP must have graduation criteria
The KEP must have an issue in the milestone

Best,
Kirsten

@kikisdeliveryservice kikisdeliveryservice added the stage/alpha Denotes an issue tracking an enhancement targeted for Alpha status label Sep 28, 2020
@kikisdeliveryservice kikisdeliveryservice added the tracked/no Denotes an enhancement issue is NOT actively being tracked by the Release Team label Oct 7, 2020
@Sh4d1 Sh4d1 mentioned this issue Nov 7, 2020
Merged
Merged
@kikisdeliveryservice
Copy link
Member

Note: this provisional kep was implemented without having a compliant kep or indicating to the enhancements team that it would be in the 1.20 milestone.

@jeremyrickard
Copy link
Contributor

jeremyrickard commented Nov 10, 2020 •
edited
Loading

Hey @Sh4d1 and @thockin,

👋 1.20 release lead here. We're really close to code freeze at this point (Thursday) and it seems like this KEP fell through the cracks and the process.

Based on this comment in the PR above: kubernetes/kubernetes#92312 (comment) can we not land this in 1.20 and instead target 1.21?

There are few things w/ the KEP that really should be address:

  • It's provisional still
  • There is no test plan
  • There is no graduation criteria

Looks like there is also a PR to update the KEP https://github.com/kubernetes/enhancements/pull/2134/files that addresses this stuff, however it also marks it as implemented and targets 1.21. Typically, we set to implemented only after the release.

Can we circle back around on this and agree that we should try to land this in 1.21 at this point? We've already got a lot of content for the release so far (sig-docs has their work cut out for themselves!!!), we're two days out from code freeze, we have these disconnects mentioned above?

@Sh4d1 Sh4d1 mentioned this issue Nov 11, 2020
Merged
@Sh4d1
Copy link
Member Author

Sh4d1 commented Nov 11, 2020 •
edited
Loading

@jeremyrickard just opened kubernetes/kubernetes#96454 to revert it, sorry again!
And updated #2134 to set it as implentable instead of implemented

@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Feb 9, 2021
@fejta-bot
Copy link

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Mar 11, 2021
@fejta-bot
Copy link

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

@k8s-ci-robot
Copy link
Contributor

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@Sh4d1
Copy link
Member Author

Sh4d1 commented Apr 10, 2021

/reopen

@k8s-ci-robot
Copy link
Contributor

@Sh4d1: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@simonhammes simonhammes mentioned this issue Aug 26, 2024
Open
@thockin thockin removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 29, 2024
@thockin
Copy link
Member

thockin commented Aug 29, 2024

@rikatz @RyanAoh - how does this look for 1.32?

@RyanAoh
Copy link
Member

RyanAoh commented Aug 29, 2024

OK, I would promote it to GA in 1.32.

@rikatz
Copy link
Contributor

rikatz commented Aug 29, 2024

Thanks @RyanAoh !

I will watch it, let me know if you need something from me!

@aojea
Copy link
Member

aojea commented Aug 29, 2024

Dan and I added way more coverage with cloud-provider-kind so I think we are good to go

@simonhammes simonhammes mentioned this issue Sep 4, 2024
Closed
1 task
@thockin
Copy link
Member

thockin commented Sep 12, 2024

Can I get a quick update? Is someone on the hook for a PR to promote this?

@thockin thockin added the lead-opted-in Denotes that an issue has been opted in to a release label Sep 12, 2024
@rikatz
Copy link
Contributor

rikatz commented Sep 12, 2024

I am not, @RyanAoh is in charge of this one :)

@RyanAoh
Copy link
Member

RyanAoh commented Sep 13, 2024 •
edited
Loading

@thockin I'm on it and I'll submit the PR(codes, docs and the PRR) this weekend.

This was referenced Sep 13, 2024
Merged
Merged
@tjons
Copy link
Contributor

tjons commented Sep 29, 2024

Hello @RyanAoh 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 11th October 2024 / 19:00 PDT Thursday 10th October 2024.

This enhancement is targeting for stage stable for v1.32 (correct me, if otherwise).

Here's where this enhancement currently stands:

  • [] KEP readme using the latest template has been merged into the k/enhancements repo.
  • KEP status is marked as implementable for latest-milestone: v1.32.
  • KEP readme has up-to-date graduation criteria
  • KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here). If your production readiness review is not completed yet, please make sure to fill the production readiness questionnaire in your KEP by the PRR Freeze deadline on Thursday 3rd October 2024 so that the PRR team has enough time to review your KEP.

To track this KEP for enhancements freeze:

  • [] the KEP readme is missing the Implementation History section. Once you update this and populate the section we can track this for enhancements freeze.

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you!

@tjons tjons moved this to At risk for enhancements freeze in 1.32 Enhancements Tracking Sep 29, 2024
@RyanAoh RyanAoh mentioned this issue Oct 1, 2024
Merged
@RyanAoh
Copy link
Member

RyanAoh commented Oct 3, 2024

@tjons The Implementation History section has been added to the kep readme.

@tjons
Copy link
Contributor

tjons commented Oct 7, 2024

@RyanAoh with all the requirements met, this enhancement is now tracked for enhancements freeze! 🚀

@tjons tjons moved this from At risk for enhancements freeze to Tracked for enhancements freeze in 1.32 Enhancements Tracking Oct 7, 2024
@hacktivist123
Copy link

Hello @RyanAoh 👋 1.32 Docs Shadow here.

Does this enhancement work planned for 1.32 require any new docs or modifications to existing docs?
If so, please follow the steps here to open a PR against the dev-1.32 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday, October 24th 2024 18:00 PDT.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.
Thank you!

@rytswd
Copy link
Member

rytswd commented Oct 16, 2024

Hi @RyanAoh 👋 -- this is Ryota (@rytswd) from the v1.32 Communications Team!

For the v1.32 release, we are currently in the process of collecting and curating a list of potential feature blogs, and we'd love for you to consider writing one for your enhancement!

As you may be aware, feature blogs are a great way to communicate to users about features which fall into (but not limited to) the following categories:

  • This introduces some breaking change(s)
  • This has significant impacts and/or implications to users
  • ...Or this is a long-awaited feature, which would go a long way to cover the journey more in detail 🎉

To opt in to write a feature blog, could you please let us know and open a "Feature Blog placeholder PR" (which can be only a skeleton at first) against the website repository by Wednesday, 30th Oct 2024? For more information about writing a blog, please find the blog contribution guidelines 📚

Tip

Some timeline to keep in mind:

  • 02:00 UTC Wednesday, 30th Oct: Feature blog PR freeze
  • Monday, 25th Nov: Feature blogs ready for review
  • You can find more in the release document

Note

In your placeholder PR, use XX characters for the blog date in the front matter and file name. We will work with you on updating the PR with the publication date once we have a final number of feature blogs for this release.

@hacktivist123
Copy link

Hello, @RyanAoh 👋 1.32 Docs Shadow here.

This is just a reminder to open a placeholder PR against dev-1.32 branch in the k/website repo for this (steps available here) for this KEP if it requires new or modifications to existing docs:

The deadline for this is Thursday, Oct 24 at 18:00 PDT.
Thanks! 🚀

@tjons
Copy link
Contributor

tjons commented Oct 27, 2024

Hello @RyanAoh 👋, Enhancements team here.

With all the implementation(code related) PRs merged as per the issue description:

This enhancement is now marked as tracked for code freeze for the 1.32 Code Freeze!

Please note that KEPs targeting stable need to have the status field marked as implemented in the kep.yaml file after code PRs are merged and the feature gates are removed.

@tjons tjons moved this from Tracked for enhancements freeze to Tracked for code freeze in 1.32 Enhancements Tracking Oct 27, 2024
@rytswd
Copy link
Member

rytswd commented Oct 29, 2024

Hi @RyanAoh 👋, v1.32 Communications Team here again!

This is a gentle reminder for the feature blog deadline mentioned above, which is 02:00 UTC Wednesday, 30th Oct. To opt in, please let us know and open a Feature Blog placeholder PR against k/website by the deadline. If you have any questions, please feel free to reach out to us!

Tip

Some timeline to keep in mind:

  • 02:00 UTC Wednesday, 30th Oct: Feature blog PR freeze
  • Monday, 25th Nov: Feature blogs ready for review
  • You can find more in the release document

Note

In your placeholder PR, use XX characters for the blog date in the front matter and file name. We will work with you on updating the PR with the publication date once we have a final number of feature blogs for this release.

@RyanAoh
Copy link
Member

RyanAoh commented Oct 29, 2024

@hacktivist123 The PR for the documentation is kubernetes/website#47938, which has already been merged.
@rytswd There's no feature blog for this update.

@rytswd
Copy link
Member

rytswd commented Oct 29, 2024

Thanks for the confirmation @RyanAoh !

@hcloud-bot hcloud-bot mentioned this issue Nov 26, 2024
Merged
lukasmetzner pushed a commit to hetznercloud/hcloud-cloud-controller-manager that referenced this issue Nov 26, 2024
@hcloud-bot
chore(main): release v1.21.0 (#806)
2316769 
<!-- section-start changelog -->
### Feature Highlights &amp; Upgrade Notes

#### Load Balancer IPs set to Private IPs

If networking support is enabled, the load balancer IPs are now
populated with the private IPs, unless the
`load-balancer.hetzner.cloud/disable-private-ingress` annotation is set
to `true`. Please make sure that you configured the annotation according
to your needs, for example if you are using `external-dns`.

#### Provided-By Label

We introduced a the label `instance.hetzner.cloud/provided-by`, which
will be automatically added to all **new** nodes. This label can have
the values `cloud` or `robot` to distinguish between our products. We
use this label in the csi-driver to ensure the daemonset is only running
on cloud nodes. We recommend to add this label to your existing nodes
with the appropriate value.

- `kubectl label node $CLOUD_NODE_NAME
instance.hetzner.cloud/provided-by=cloud`
- `kubectl label node $ROBOT_NODE_NAME
instance.hetzner.cloud/provided-by=robot`

#### Load Balancer IPMode Proxy

Kubernetes KEP-1860 added a new field to the Load Balancer Service
Status that allows us to mark if the IP address we add should be
considered as a Proxy (always send traffic here) and VIP (allow
optimization by keeping the traffic in the cluster).

Previously Kubernetes considered all IPs as VIP, which caused issues
when when the PROXY protocol was in use. We have previously recommended
to use the annotation `load-balancer.hetzner.cloud/hostname` to
workaround this problem.

We now set the new field to `Proxy` if the PROXY protocol is active so
the issue should no longer appear. If you only added the
`load-balancer.hetzner.cloud/hostname` annotation for this problem, you
can remove it after upgrading.

Further information:

- kubernetes/enhancements#1860
-
#160 (comment)

### Features

- **service**: Specify private ip for loadbalancer (#724)
- add support &amp; tests for Kubernetes 1.31 (#747)
- **helm**: allow setting extra pod volumes via chart values  (#744)
- **instance**: add label to distinguish servers from Cloud and Robot
(#764)
- emit event when robot server name and node name mismatch (#773)
- **load-balancer**: Set IPMode to &#34;Proxy&#34; if load balancer is
configured to use proxy protocol (#727) (#783)
- **routes**: emit warning if cluster cidr is misconfigured (#793)
- **load-balancer**: ignore nodes that don&#39;t use known provider IDs
(#780)
- drop tests for kubernetes v1.27 and v1.28

### Bug Fixes

- populate ingress private ip when disable-private-ingress is false
(#715)
- wrong version logged on startup (#729)
- invalid characters in label instance-type of robot servers (#770)
- no events are emitted as broadcaster has no sink configured (#774)

### Kubernetes Support

This version was tested with Kubernetes 1.29 - 1.31. Furthermore, we
dropped v1.27 and v1.28 support.

<!-- section-end changelog -->

---

<details>
<summary><h4>PR by <a
href="https://github.com/apricote/releaser-pleaser">releaser-pleaser</a>
🤖</h4></summary>

If you want to modify the proposed release, add you overrides here. You
can learn more about the options in the docs.

## Release Notes

### Prefix / Start

This will be added to the start of the release notes.

```rp-prefix
### Feature Highlights & Upgrade Notes

#### Load Balancer IPs set to Private IPs

If networking support is enabled, the load balancer IPs are now populated with the private IPs, unless the `load-balancer.hetzner.cloud/disable-private-ingress` annotation is set to `true`. Please make sure that you configured the annotation according to your needs, for example if you are using `external-dns`.

#### Provided-By Label

We introduced a the label `instance.hetzner.cloud/provided-by`, which will be automatically added to all **new** nodes. This label can have the values `cloud` or `robot` to distinguish between our products. We use this label in the csi-driver to ensure the daemonset is only running on cloud nodes. We recommend to add this label to your existing nodes with the appropriate value.

- `kubectl label node $CLOUD_NODE_NAME instance.hetzner.cloud/provided-by=cloud`
- `kubectl label node $ROBOT_NODE_NAME instance.hetzner.cloud/provided-by=robot`

#### Load Balancer IPMode Proxy

Kubernetes KEP-1860 added a new field to the Load Balancer Service Status that allows us to mark if the IP address we add should be considered as a Proxy (always send traffic here) and VIP (allow optimization by keeping the traffic in the cluster).

Previously Kubernetes considered all IPs as VIP, which caused issues when when the PROXY protocol was in use. We have previously recommended to use the annotation `load-balancer.hetzner.cloud/hostname` to workaround this problem.

We now set the new field to `Proxy` if the PROXY protocol is active so the issue should no longer appear. If you  only added the `load-balancer.hetzner.cloud/hostname` annotation for this problem, you can remove it after upgrading.

Further information:
- kubernetes/enhancements#1860
- #160 (comment)
```

### Suffix / End

This will be added to the end of the release notes.

```rp-suffix
### Kubernetes Support

This version was tested with Kubernetes 1.29 - 1.31. Furthermore, we dropped v1.27 and v1.28 support.
```

</details>

Co-authored-by: releaser-pleaser <>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@thockin thockin

@rikatz rikatz

@RyanAoh RyanAoh

Labels
lead-opted-in Denotes that an issue has been opted in to a release sig/network Categorizes an issue or PR as relevant to SIG Network. stage/beta Denotes an issue tracking an enhancement targeted for Beta status
Projects
1.29 Enhancements Tracking
Status: Tracked for Code Freeze
1.30 Enhancements Tracking
Status: Tracked for Doc Freeze
1.32 Enhancements Tracking
Status: Tracked for code freeze
SIG Network KEPs
Status: Beta
Milestone
v1.32
Development

No branches or pull requests