Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Warp sync state machine refactoring #1060

Merged
merged 62 commits into from
Sep 21, 2023
Merged
Changes from 1 commit
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
eca8295
Use let-else construct
tomaka Aug 22, 2023
b780aae
Add sources_by_finalized_height
tomaka Aug 22, 2023
1dd8f1a
Allow querying runtime and chain information from any source
tomaka Aug 22, 2023
9b722c1
Add comment
tomaka Aug 23, 2023
2bc3b34
Implement Index<RequestId>
tomaka Aug 23, 2023
73a3220
Merge branch 'main' into warp-sync-refactor
tomaka Aug 30, 2023
e868c65
Merge `DownloadFragments` and `PendingVerify` stages
tomaka Sep 1, 2023
bbb4015
Bugfix with state inconsistency
tomaka Sep 1, 2023
94a5f6d
Download warp sync fragments in parallel of veryfing the previous batch
tomaka Sep 1, 2023
ec0a3f2
Merge Phase::RuntimeDownload with Phase::ChainInformationDownload
tomaka Sep 2, 2023
fddd72c
Remove `Phase` enum and move everything to base state machine
tomaka Sep 3, 2023
398de82
Small comment tweaks
tomaka Sep 3, 2023
fea7871
Merge branch 'main' into warp-sync-refactor
tomaka Sep 5, 2023
03ce8cd
Verify fragments even if source has disconnected
tomaka Sep 5, 2023
0d48d4e
Small refactor
tomaka Sep 5, 2023
fd35484
Implement correctly fetching from proof
tomaka Sep 5, 2023
52b263c
Fix wasm build
tomaka Sep 5, 2023
45f5fff
Embed the verifier within warp_sync.rs
tomaka Sep 5, 2023
f58103b
Add TODO
tomaka Sep 5, 2023
ccb9568
Also check the justification target number
tomaka Sep 5, 2023
19942f0
PR link and more changelog
tomaka Sep 5, 2023
c9c7cca
Polish the fragment verification error
tomaka Sep 6, 2023
e05244a
Add `Config::pause_if_blocks_gap_lower_than`
tomaka Sep 6, 2023
4f50871
Merge branch 'main' into warp-sync-refactor
tomaka Sep 6, 2023
9910876
`WarpSync` no longer stops after success
tomaka Sep 7, 2023
913726e
Merge branch 'main' into warp-sync-refactor
tomaka Sep 7, 2023
061039f
Rename `start_chain_information`
tomaka Sep 7, 2023
1e28edf
Move runtime calls to main state machine for parallelism
tomaka Sep 8, 2023
d961862
Merge branch 'main' into warp-sync-refactor
tomaka Sep 8, 2023
ed588f4
CHANGELOG update
tomaka Sep 8, 2023
1274f02
Merge branch 'main' into warp-sync-refactor
tomaka Sep 8, 2023
4a8ab39
Don't warp sync the same block if there's no point in doing that
tomaka Sep 8, 2023
4920d73
Remove `already_tried` system
tomaka Sep 11, 2023
db05517
Move things around
tomaka Sep 11, 2023
3758f17
Merge branch 'main' into warp-sync-refactor
tomaka Sep 11, 2023
28d8bcf
Store the header in scale-encoded form and cache some fields
tomaka Sep 11, 2023
5cd7416
Merge branch 'main' into warp-sync-refactor
tomaka Sep 13, 2023
5f12473
Docfix
tomaka Sep 13, 2023
f96892f
Merge branch 'main' into warp-sync-refactor
tomaka Sep 20, 2023
24f0302
Small doc tweaks
tomaka Sep 20, 2023
a3172b6
Don't download call proofs if there are fragments to verify
tomaka Sep 20, 2023
f7aa125
Add TODO
tomaka Sep 20, 2023
b7efa99
Add TODO
tomaka Sep 20, 2023
bd6d159
Better reporting on fragment verification
tomaka Sep 20, 2023
9d83112
Make sure that headers are above the requested height
tomaka Sep 20, 2023
20f3754
Fix the calculation of the warp sync minimum gap
tomaka Sep 20, 2023
3c11ea3
Docfix
tomaka Sep 20, 2023
6ad0fc6
Index requests by source to remove a `O(n)` operation
tomaka Sep 20, 2023
6aeae88
Update the finalized block of the source when receiving a response
tomaka Sep 20, 2023
129cbd3
Remove TODO
tomaka Sep 20, 2023
b8c48bc
Option<Error> to Result<(), Error>
tomaka Sep 20, 2023
f4c1edc
Add TODO
tomaka Sep 20, 2023
63d6e4a
Download the runtime and call proofs only once there's no fragment do…
tomaka Sep 20, 2023
96da56f
Update warp sync state machine on commit message
tomaka Sep 20, 2023
e0bf656
Fix `sources_by_finalized_height` state mismatch
tomaka Sep 20, 2023
ce0ac6b
Add TODO
tomaka Sep 20, 2023
7ac62fc
Punish sources that send back invalid data
tomaka Sep 21, 2023
75ee2a2
Punish sources, part 2
tomaka Sep 21, 2023
8416988
Adjust code that handles empty lists of fragments
tomaka Sep 21, 2023
22aacbb
Add TODO
tomaka Sep 21, 2023
26e0f12
Move things around
tomaka Sep 21, 2023
7b38698
Tweak error types
tomaka Sep 21, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update warp sync state machine on commit message
tomaka committed Sep 20, 2023

Unverified

This user has not yet uploaded their public signing key.
commit 96da56fdc9849e1ed98c68f8af5dd6f07475c1a5
18 changes: 17 additions & 1 deletion lib/src/sync/all.rs
Original file line number Diff line number Diff line change
@@ -33,6 +33,7 @@
use crate::{
chain::{blocks_tree, chain_information},
executor::host,
finality::grandpa,
header,
sync::{all_forks, optimistic, warp_sync},
trie::Nibble,
@@ -1381,11 +1382,26 @@ impl<TRq, TSrc, TBl> AllSync<TRq, TSrc, TBl> {
}
}
}
(
AllSyncInner::GrandpaWarpSync { inner, .. },
SourceMapping::GrandpaWarpSync(source_id),
) => {
let block_number = match grandpa::commit::decode::decode_grandpa_commit(
&scale_encoded_message,
inner.block_number_bytes(),
) {
Ok(msg) => msg.message.target_number,
Err(_) => return GrandpaCommitMessageOutcome::Discarded,
};

inner.set_source_finality_state(*source_id, block_number);
GrandpaCommitMessageOutcome::Discarded
}
(AllSyncInner::Optimistic { .. }, _) => GrandpaCommitMessageOutcome::Discarded,
(AllSyncInner::GrandpaWarpSync { .. }, _) => GrandpaCommitMessageOutcome::Discarded,

// Invalid internal states.
(AllSyncInner::AllForks(_), _) => unreachable!(),
(AllSyncInner::GrandpaWarpSync { .. }, _) => unreachable!(),
(AllSyncInner::Poisoned, _) => unreachable!(),
}
}