Concurrently reconcile CloudStackMachine resources #290
Conversation
|
Skipping CI for Draft Pull Request. |
k8s-ci-robot
added
do-not-merge/work-in-progress
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: chrisdoherty4 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
✅ Deploy Preview for kubernetes-sigs-cluster-api-cloudstack ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
k8s-ci-robot
added
approved
chrisdoherty4
force-pushed
the
feature/concurrenc-reconciles
branch
2 times, most recently
from
2d5efba to
37a85e8
Compare
k8s-ci-robot
added
size/M
chrisdoherty4
force-pushed
the
feature/concurrenc-reconciles
branch
from
37a85e8 to
4fb8931
Compare
Codecov ReportPatch coverage has no change and project coverage change:
Additional details and impacted files@@ Coverage Diff @@
## main #290 +/- ##
==========================================
- Coverage 25.29% 25.25% -0.05%
==========================================
Files 59 59
Lines 5585 5582 -3
==========================================
- Hits 1413 1410 -3
Misses 4035 4035
Partials 137 137
☔ View full report in Codecov by Sentry. |
chrisdoherty4
force-pushed
the
feature/concurrenc-reconciles
branch
from
4fb8931 to
9f73dae
Compare
chrisdoherty4
changed the title
k8s-ci-robot
removed
the
do-not-merge/work-in-progress
|
/run-e2e -c 4.18 |
|
/lgtm |
k8s-ci-robot
added
the
lgtm
|
/hold |
k8s-ci-robot
added
the
do-not-merge/hold
|
/hold |
|
The E2E don't seem to be getting kicked off? /assign @vishesh92 @weizhouapache |
|
@chrisdoherty4: GitHub didn't allow me to assign the following users: vishesh92. Note that only kubernetes-sigs members with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
@chrisdoherty4 it's possible the backend BO script is facing Github rate limits (we found recently the script/jenkins jobs are running but couldn't post results as Github rate limit would void the API request to post comments for some reason) /run-e2e help |
|
@rohityadavcloud Usage: /run-e2e [-k Kubernetes_Version] [-c CloudStack_Version] [-h Hypervisor] [-i Template/Image] [-f Kubernetes_Version_Upgrade_From] [-t Kubernetes_Version_Upgrade_To]
Examples:
|
|
/run-e2e -c 4.18 |
|
@rohityadavcloud a jenkins job has been kicked to run test with following paramaters:
|
|
Test Results : (tid-272) |
|
/run-e2e -c 4.18 |
|
@chrisdoherty4 a jenkins job has been kicked to run test with following paramaters:
|
|
/uncc @davidjumani |
|
Test Results : (tid-273) |
|
The failing affinity E2E is also failing on main so is not an error introduced by this change. |
|
/unhold |
k8s-ci-robot
removed
the
do-not-merge/hold
AWS analyzed CAPC in high node count contexts and found it takes considerable time to scale clusters. Part of the issue stems from CloudStackMachine resources being reconciled serially. This change enables concurrent reconciliation of CloudStackMachine resources improving the efficiency and preventing other parts of the system from reacting to slowness.
I have tested these changes by scaling up and down a machine deployment from 1 to 11 nodes. Scale ups took comparable times (55s) vs serial reconciliation which is expected as most of the time is consumed by VM provisioning. Scale down had an 85% improvement from 1m57s to 27s.
Related #274