BEEFY: add BEEFY-specific warp proofs so we can warp sync when using BEEFY

[acatangiu]

Right now BEEFY doesn't work with warp sync.

We should add BEEFY warp proofs so that BEEFY can properly initialize when doing a warp sync.

We could then further enhance warp syncing to be able to sync exclusively using BEEFY warp-proofs. Light clients could warp sync without any GRANDPA proofs.

[Polkadot-Forum]

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/warp-sync-and-beefy/4616/1

[Polkadot-Forum]

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/polkadot-digest-30-nov-2023/5010/1

[acatangiu]

Discussed this with @andresilva and @serban300 and we came up with following plan:

• change GRANDPA warp sync protocol name from /<genesis_hash>/sync/warp to /<genesis_hash>/sync/warp/grandpa (while still supporting old name too),
• create new sync protocol /<genesis_hash>/sync/warp/beefy/ecdsa which will be a separate instance (aka both can run in parallel)
• switch substrate nodes to using sync/warp/beefy/ecdsa when doing --sync warp - nodes will warp sync using BEEFY
• deprecate but still support providing GRANDPA WarpSyncProofs on sync/warp/grandpa protocol for light-clients (e.g. smoldot) while they (potentially) switch to BEEFY warp proofs

Later this year in 2024, when we get BLS support in BEEFY, create new protocol that will replace others (again with deprecation period): /<genesis_hash>/sync/warp/beefy/bls. Once we have BEEFY-BLS, warp proofs will be orders of magnitude smaller than current GRANDPA ones and everyone will want to move to that.

cc @tomaka

[acatangiu]

We'll go through RFC for this and we can refine it further there.

[tomaka]

Sounds overall good to me, but I'm confused by:

deprecate but still support providing GRANDPA WarpSyncProofs on sync/warp/grandpa protocol for light-clients (e.g. smoldot) while they (potentially) switch to BEEFY warp proofs

Is the idea that in the long term, you would do only a BEEFY warp sync, then download or get gossiped the Grandpa commits/justifications for the blocks between the BEEFY head and the GRANDPA head?

With the removal of the GRANDPA warp sync, the light client couldn't know anymore which block has a GRANDPA justification and which hasn't, which means that it would have to download every block between the two heads before being able to know what is finalized.
This isn't a problem by itself, but it might mean a longer initialization time, as the warp sync no longer propulsates you directly to where you want but a gap of unknown size behind.

Overall this increase in loading time would probably be compensated by the BLS signatures, but it's not totally clear to me that it's a good idea to get rid of GRANDPA warp syncing altogether.

EDIT: We could also make the light client lie to the API user and pretend that only the BEEFY head is finalized while it still tries to find out which block is the GRANDPA head, as the outcome is the same.

Later this year,

Do you mean 2023? If it's soon then I don't see the point in having ECDSA first? But I guess you mean 2024?

[acatangiu]

Do you mean 2023? If it's soon then I don't see the point in having ECDSA first? But I guess you mean 2024?

sorry, yes, I mean second part of 2024.

With the removal of the GRANDPA warp sync, the light client couldn't know anymore which block has a GRANDPA justification and which hasn't.

I believe a possible end goal is for light clients to depend on BEEFY alone and not care about GRANDPA - GRANDPA would be a low(er) level consensus mechanism used only by full nodes - but anything in the light-clients realm would just use BEEFY for proving finality and not even need to be aware of existence of GRANDPA.

The benefits of that is simpler light-clients, cheaper and faster syncing (especially with BLS) - costs being development cost (light-client switch from GRANDPA to BEEFY proofs), and taking a finality "latency" hit as BEEFY runs a few blocks behind GRANDPA.

[tomaka]

and taking a finality "latency" hit as BEEFY runs a few blocks behind GRANDPA.

Since light clients cannot trust non-finalized blocks, this it adds even more latency between the moment when the user does something and the moment when that thing happens "live", as "live" is the finalized block.

So I'm not completely sold on the idea that light clients (I'm talking about the ones that have a lot of memory and CPU power) shouldn't take Grandpa into account.
But I'm happy with the solution of ending the light client initialization right after at the end of the BEEFY warp sync, and initially pretend that the BEEFY head is the head of the chain, and then only afterwards download the GRANDPA finalized blocks.

[nicolasochem]

Was this fixed? Should this be closed?

https://github.com/paritytech/polkadot-sdk/blob/master/prdoc/1.6.0/pr_2689.prdoc

[serban300]

No, we should keep this one opened. The issues are related, but they differ. The PR that you linked removed the limitation of not being able to run BEEFY when GRANDPA warp sync is enabled. This issue is about being able to use BEEFY warp sync proof instead of GRANDPA warp sync proofs. It would be an optimization.

[nicolasochem]

No, we should keep this one opened. The issues are related, but they differ. The PR that you linked removed the limitation of not being able to run BEEFY when GRANDPA warp sync is enabled. This issue is about being able to use BEEFY warp sync proof instead of GRANDPA warp sync proofs. It would be an optimization.

Thanks, can you clarify though, whether warp-syncing a Kusama validator is ok on 1.0.7?

If I warp-sync a fresh kusama validator now, and rotate all keys (including beefy keys), will all validator duties be performed correctly?

[acatangiu]

Thanks, can you clarify though, whether warp-syncing a Kusama validator is ok on 1.0.7?

Warp syncing a Kusama validator is ok on 1.7.0.

If I warp-sync a fresh kusama validator now, and rotate all keys (including beefy keys), will all validator duties be performed correctly?

Yes (as long as you use 1.6.x or 1.7.x node version to do the warp-sync - ideally clean DB too before).

[Polkadot-Forum]

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/polkadot-digest-21-feb-2024/6358/1