[Bug] Removed subgraph does not get removed until graph node restart

[paymog]

Bug report

I don't know how to reliably reproduce this, but I'm finding that on v0.33.0 often times when a subgraph gets removed, it doesn't actually stop indexing until the graph node restarts. I suspect this is related to #5011.

I'm running a multi node cluster (this has happened in a two node cluster and a 10 node cluster) with sharding. So far I've only seen this issue happen with subgraphs that are on firehose enabled chains that are erroring.

I will keep this ticket updated as I find more info.

Relevant log output

No response

IPFS hash

No response

Subgraph name or link to explorer

No response

Some information to help us out

• Tick this box if this bug is caused by a regression found in the latest release.
• Tick this box if this bug is specific to the hosted service.
• I have searched the issue tracker to make sure this issue is not a duplicate.

OS information

None

[azf20]

Thanks @paymog - I don't know if this will be provider related. As this is a multi node cluster, maybe the issue is flakiness at that level, or do you see this consistently (i.e. never stops indexing, or sometimes doesn?)

[paymog]

To be honest I only saw this one time and have no idea how to reproduce it. Will definitely let you know if this happens again.

[azf20]

OK - will close for now, please re-open if it does, thanks!