Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fast catchup #1023

Merged
merged 6 commits into from
Jun 13, 2022
Merged

fast catchup #1023

merged 6 commits into from
Jun 13, 2022

Conversation

shiqizng
Copy link
Contributor

@shiqizng shiqizng commented Jun 9, 2022

Summary

this PR adds fast catchup method

Test Plan

testing it manually.

opts.IndexerDatadir = indexerDataDir
opts.AlgodDataDir = algodDataDir
opts.AlgodToken = algodToken
opts.AlgodAddr = algodAddr
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why these are showing up as diffs.

@codecov
Copy link

codecov bot commented Jun 9, 2022

Codecov Report

❗ No coverage uploaded for pull request base (localledger/integration@45b9dbe). Click here to learn what that means.
The diff coverage is n/a.

@@                    Coverage Diff                     @@
##             localledger/integration    #1023   +/-   ##
==========================================================
  Coverage                           ?   59.24%           
==========================================================
  Files                              ?       47           
  Lines                              ?     8722           
  Branches                           ?        0           
==========================================================
  Hits                               ?     5167           
  Misses                             ?     3070           
  Partials                           ?      485           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 45b9dbe...ed4bd11. Read the comment docs.

node, err := node.MakeFull(
logging.NewLogger(),
opts.IndexerDatadir,
algodConfig.AutogenLocal,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you happen to try this configuration for testnet/betanet?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have only tested it for testnet so far.

Copy link
Contributor

@winder winder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two small suggestions

@shiqizng shiqizng merged commit eff4e6f into localledger/integration Jun 13, 2022
@shiqizng shiqizng deleted the shiqi/fastcatchup branch June 16, 2022 20:49
Eric-Warehime added a commit that referenced this pull request Jul 21, 2022
* Local Ledger (#1011)

* integrate block processor

* Local Ledger Deployment (#1013)

* add simple local ledger migration

* add deleted opts

* fast catchup (#1023)

* add fast catchup

* Localledger merge (#1036)

* return empty lists from fetchApplications and fetchAppLocalStates (#1010)

* Update model to converge with algod (#1005)

* New Feature: Adds Data Directory Support (#1012)

- Updates the go-algorand submodule hash to point to rel/beta
- Moves the cpu profiling file, pid file and indexer configuration file
  to be options of only the daemon sub-command
- Changes os.Exit() to be a panic with a special handler.  This is so
  that defer's are handled instead of being ignored.
- Detects auto-loading configuration files in the data directory and
  issues errors if equivalent command line arguments are supplied.
- Updates the README with instructions on how to use the auto-loading
  configuration files and the data directory.

* Update mockery version

Co-authored-by: erer1243 <[email protected]>
Co-authored-by: AlgoStephenAkiki <[email protected]>

* recovery scenario (#1024)

* handle ledger recovery scenario

* refactor create genesis block (#1026)

* refactor create genesis block

* Adds Local Ledger Readme (#1035)

* Adds Local Ledger Readme

Resolves #4109

Starts Readme docs

* Update docs/LocalLedger.md

Co-authored-by: Will Winder <[email protected]>

* Update docs/LocalLedger.md

Co-authored-by: Will Winder <[email protected]>

* Update docs/LocalLedger.md

Co-authored-by: Will Winder <[email protected]>

* Removed troubleshooting section

Co-authored-by: Will Winder <[email protected]>

* update ledger file path and migration (#1042)

* LocalLedger Refactoring + Catchpoint Service (#1049)

Part 1

    cleanup genesis file access.
    put node catchup into a function that can be swapped out with the catchup service.
    pass the indexer logger into the block processor.
    move open ledger into a util function, and move the initial state util function into a new ledger util file.
    add initial catchupservice implementation.
    move ledger init from daemon.go to constructor.
    Merge multiple read genesis functions.

Part 2

    Merge local_ledger migration package into blockprocessor.
    Rename Migration to Initialize
    Use logger in catchup service catchup

Part 3

    Update submodule and use NewWrappedLogger.
    Make util.CreateInitState private

* build: merge develop into localledger/integration (#1062)

* Ledger init status (#1058)

* Generate an error if the catchpoint is not valid for initialization. (#1075)

* Use main logger in handler and fetcher. (#1077)

* Switch from fullNode catchup to catchpoint catchup service. (#1076)

* Refactor daemon, add more tests (#1039)

Refactors daemon cmd into separate, testable pieces.

* Merge develop into localledger/integration (#1083)

* Misc Local Ledger cleanup (#1086)

* Update processor/blockprocessor/initialize.go

Co-authored-by: Zeph Grunschlag <[email protected]>

* commit

* fix function call args

* RFC-0001: Rfc 0001 impl (#1069)

Adds an Exporter interface and a noop exporter implementation with factory methods for construction

* Fix test errors

* Add/fix tests

* Add postgresql_exporter tests

* Update config loading

* Change BlockExportData to pointers

* Move and rename ExportData

* Add Empty func to BlockData

* Add comment

Co-authored-by: shiqizng <[email protected]>
Co-authored-by: [email protected] <[email protected]>
Co-authored-by: erer1243 <[email protected]>
Co-authored-by: AlgoStephenAkiki <[email protected]>
Co-authored-by: Will Winder <[email protected]>
Co-authored-by: Zeph Grunschlag <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants