Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add exponential backoff retry for non deterministic errors #2988

Merged
merged 5 commits into from
Dec 7, 2021

Conversation

evaporei
Copy link
Contributor

@evaporei evaporei commented Nov 19, 2021

After extensive testing, I think this is finally correct 🙌

This PR adds an exponential backoff to keep retrying non-deterministic errors until they get solved and the subgraph unfails.

Main changes:

Solves #2945

@evaporei evaporei requested a review from leoyvens November 19, 2021 17:19
@evaporei evaporei self-assigned this Nov 19, 2021
core/src/subgraph/instance_manager.rs Outdated Show resolved Hide resolved
core/src/subgraph/instance_manager.rs Outdated Show resolved Hide resolved
@evaporei evaporei force-pushed the otavio/retry-non-deterministic-errors branch 2 times, most recently from 1b0937b to c2de449 Compare November 22, 2021 15:04
@evaporei evaporei force-pushed the otavio/retry-non-deterministic-errors branch 2 times, most recently from 30420ed to 910f7bd Compare November 25, 2021 19:04
@evaporei evaporei force-pushed the otavio/retry-non-deterministic-errors branch 2 times, most recently from c2ce45a to 34a77b7 Compare November 30, 2021 20:16
@evaporei evaporei requested a review from leoyvens November 30, 2021 20:25
store/postgres/src/deployment_store.rs Outdated Show resolved Hide resolved
@evaporei evaporei force-pushed the otavio/retry-non-deterministic-errors branch from 34a77b7 to c0e4879 Compare December 6, 2021 18:43
@evaporei evaporei merged commit 5ddc3e1 into master Dec 7, 2021
@evaporei evaporei deleted the otavio/retry-non-deterministic-errors branch December 7, 2021 15:05
evaporei added a commit that referenced this pull request Mar 3, 2022
* deployment: Have just one is_failed function for SubgraphHealth

* writable_store: Add method to get health of deployment

* instance_manager: Use subgraph health to decide unfailure

* instance_manager: Add exponential backoff retry for non-deterministic errors

* store: Make health function async
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants