Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restart cr-syncer informers on too many subsequent conflict errors #69

Merged
merged 2 commits into from
Aug 27, 2021

Conversation

oliver-goetz
Copy link
Contributor

Following up to this issue #64

When trying to run cloud-robotics with Istio ingresses I realised that is is hard to guarantee that no issues as in #64 happen again as long as client-go does not identify connection drops reliably.
One way to identify that a watcher died are many subsequent conflict errors when cr-syncers updates the remote cluster

2021/01/26 16:40:16 Syncing key "default/1710.2053438" from queue "downstream" failed: WarehouseOrder default/1710.2053438 @ 184236994: update status failed: Operation cannot be fulfilled on warehouseorders.ewm.sap.com "1710.2053438": the object has been modified; please apply your changes to the latest version and try again

This change restarts informers for one CRD in case 5 subsequent conflict errors occur. This is more efficient than restarting the whole cr-syncer because it could happen for single CRDs.

@google-cla google-cla bot added the cla: yes cla signed label Jul 22, 2021
@oliver-goetz
Copy link
Contributor Author

Hi @ensonic, a friendly reminder

@drigz
Copy link
Contributor

drigz commented Aug 23, 2021

I'm afraid Stefan is on vacation at the moment, but he'll be back later this week. Let me know if this is urgent and I can try to take over.

@oliver-goetz
Copy link
Contributor Author

thanks for notifying me.
It is not urgent. I'm just back from my vacation an checking what happened in the meantime 😄

@drigz
Copy link
Contributor

drigz commented Aug 23, 2021

Ah, in that case, I think the answer is "some walks in the Austrian Alps" but Stefan can confirm :)

@oliver-goetz
Copy link
Contributor Author

that sounds good 😄

@ensonic ensonic merged commit 52f5917 into googlecloudrobotics:master Aug 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes cla signed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants