Add a checkpoint warp sync system to the full node

[tomaka]

paritytech/smoldot#165

[tomaka]

Requires #166 because Polkadot (and presumable other chains) uses child tries in its recent blocks.

[tomaka]

Now that #166 is done, we can progress here.

The design issues are the same as in Substrate: is it a good idea to store a partially-downloaded warp sync chain storage in the database? If the node stops for a long time, it is likely that we can't find any node capable of serving the chain storage that we have partially downloaded anymore.
Same problem if we take too long to download the warp sync storage.

[tomaka]

The design issues are the same as in Substrate: is it a good idea to store a partially-downloaded warp sync chain storage in the database? If the node stops for a long time, it is likely that we can't find any node capable of serving the chain storage that we have partially downloaded anymore.
Same problem if we take too long to download the warp sync storage.

A solution to this problem is that the full node could download and execute blocks on top of the finalized block while it is still downloading the state of the finalized block, using storage proofs if during the execution some items are missing.

Once a block has been executed, we know that any item not modified during the execution is identical between the parent block and the child block. If the storage of the parent block becomes inaccessible, we can still continue by using the storage of the child block instead.

Consequently, I think that the full node should simply use the same warp syncing code as the light client, and then download the storage asynchronously from one of its high level services.

[tomaka]

(I had a branch almost ready that adds a separate mode to the warp syncing algorithm that downloads the entire storage, but I'm throwing it away now)

[tomaka]

Changes that need to be made:

• The database should give the possibility to store a partial state.
• If, when executing a block, the consensus service requires a storage entry that is missing from the database, it downloads a call proof from the network instead. The items in the call proof are inserted in the database.
• In parallel of the consensus service, a new "state building service" should download the missing storage entries. Later, this "state building service" can also be used to download older blocks that are missing from the database, and, in the case of an archive node, execute older blocks to build their state.

[tomaka]

The last feature that is missing is the possibility to not verify a certain block yet (as we're currently downloading some items that we're missing from the storage of its parent).
In terms of actual logic, it's pretty easy, as we have a list of leaves ready to be verified, and all we need is a way to add an additional filter. But it's not clear to me how to actually do that in terms of API.