Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Design statement distribution subsystem #4195

Open
2 of 3 tasks
axaysagathiya opened this issue Sep 23, 2024 · 6 comments · May be fixed by #4379
Open
2 of 3 tasks

Design statement distribution subsystem #4195

axaysagathiya opened this issue Sep 23, 2024 · 6 comments · May be fixed by #4379
Assignees
Labels
S-subsystems-backing issues related to polkadot host backing subsystem functionality. T-research this issue/pr is a research type issue.

Comments

@axaysagathiya
Copy link
Contributor

axaysagathiya commented Sep 23, 2024

Issue summary

This parachain subsystem is responsible for

  • distributing signed statements that we have generated and
  • forwarding statements generated by our peers

Acceptance criteria

  • read host implementors guide
  • Observe rust code and figure out what we need to implement
  • Understand Rust code and raise issues for all the necessary tasks.
@axaysagathiya axaysagathiya added S-subsystems-backing issues related to polkadot host backing subsystem functionality. T-investigation this issue/pr is an investigation, probably related to some bug with unknown causes. labels Sep 23, 2024
@axaysagathiya
Copy link
Contributor Author

There are 2 versions of the statement distribution protocol.

  • v1 is for before async backing support
  • v2 is designed for async backing

@axaysagathiya
Copy link
Contributor Author

axaysagathiya commented Sep 23, 2024

Statement distribution subsystem V2

  • Validators must have statements, candidates, and persisted validation from all other validators.
  • phases of statement distribution subsystem design
    • in group distribution - cluster - for legitimate fresh candidates
    • out group distribution - grid - to quickly get backed candidates in many validators
  • validators request candidates (validated) lazily through request/response protocols.
    • can request the full CommittedCandidateReceipt and PersistedValidationData, along with statements

@axaysagathiya
Copy link
Contributor Author

Statement distribution subsystem V1

  • On receiving StatementDistributionMessage::Share we make sure to send messages to our backing group in addition to random other peers, to ensure a fast backing process and getting all statements quickly for distribution.

  • This subsystem tracks equivocating validators and stops accepting information from them. It establishes a data-dependency order:

    • In order to receive a Seconded message we have the corresponding chain head in our view
    • In order to receive a Valid message we must have received the corresponding Seconded message
  • This subsystem is responsible for checking message signatures.

Peer Receipt State Machine

A: Initial State. Receive SignedFullStatement(Statement::Second): extract Statement, forward to Candidate Backing, proceed to B. Receive any other SignedFullStatement variant: drop it.

B: Receive any SignedFullStatement: check signature and determine whether the statement is new to us. if new, forward to Candidate Backing and circulate to other peers. Receive OverseerMessage::StopWork: proceed to C.

C: Receive any message for this block: drop it.

Large statements

  • The receiver of large statement message needs to request the actual payload via request/response by means of a StatementFetchingV1 request.

@axaysagathiya
Copy link
Contributor Author

axaysagathiya commented Sep 30, 2024

To implement v1, we need to handle the main 3 kinds of messages.

  1. messages received from other subsystems
    • share signed statements with other validators
    • notification of candidate being backed
    • network bridge update
  2. request the large statements from the other validators
    • message to send a request to other validators
    • message to get an update of available peers to try fetching statements
    • report a peer who behaves worse than just not providing data
    • fetched finished
  3. respond to the large statement to the other validators
    • check if we have data and send it back to the requester

@axaysagathiya
Copy link
Contributor Author

We need to have a separate Goroutine that will simultaneously process the requests we get from other validators to fetch the large statement for a candidate.

This Goroutine should ensure we process only some predecided number of requests simultaneously.
Process the message and send back the large statement to the requester

@timwu20 timwu20 changed the title Investigate on statement distribution subsystem Investigate statement distribution subsystem Oct 24, 2024
@P1sar P1sar removed the T-investigation this issue/pr is an investigation, probably related to some bug with unknown causes. label Nov 25, 2024
@P1sar P1sar changed the title Investigate statement distribution subsystem Design statement distribution subsystem Nov 25, 2024
@P1sar P1sar changed the title Design statement distribution subsystem Investigate statement distribution subsystem Nov 25, 2024
@P1sar
Copy link
Member

P1sar commented Nov 25, 2024

@EclesioMeloJunior add questions that left to be answered

@P1sar P1sar changed the title Investigate statement distribution subsystem DEsign statement distribution subsystem Nov 26, 2024
@P1sar P1sar changed the title DEsign statement distribution subsystem Design statement distribution subsystem Nov 26, 2024
@P1sar P1sar added T-research this issue/pr is a research type issue. S-subsystems-availability issues related to polkadot host availability subsystem functionality. S-subsystems-backing issues related to polkadot host backing subsystem functionality. and removed S-subsystems-backing issues related to polkadot host backing subsystem functionality. S-subsystems-availability issues related to polkadot host availability subsystem functionality. labels Dec 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-subsystems-backing issues related to polkadot host backing subsystem functionality. T-research this issue/pr is a research type issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants