Skip to content

Commit

Permalink
docs: add snapshot instructions (#224)
Browse files Browse the repository at this point in the history 
* docs: add snapshot instructions

* docs: add snapshot instructions

* fix: use x-goog-hash rather than etag

* fix: specify 0.10.3
  • Loading branch information
@Teasel-Ian
Teasel-Ian authored Apr 29, 2022
1 parent 8e4dbf6 commit e7b7407
Show file tree
Hide file tree
Showing 4 changed files with 73 additions and 17 deletions.
2 changes: 1 addition & 1 deletion docs/docs/joining-a-testnet.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ curl https://rpc-capricorn.fetch.ai:443 | jq '.result.genesis' > ~/.fetchd/confi
fetchd start --p2p.seeds=fec822ecf6e503a694a709ce663fd0c6da5fda3e@connect-capricorn.fetch.ai:36956
```

Your local node will then start to synchronise itself with the network, replaying all blocks and transactions up to the current block. Depending on the age of the network and your hard disk speed, this could take a while.
Your local node will then start to synchronise itself with the network, replaying all blocks and transactions up to the current block. Depending on the age of the network and your hard disk speed, this could take a while. Consider using [chain snapshots](../snapshots/) to speed up this process.

To know when your node as finished syncing, you can query it's status from its RPC API:

Expand Down
35 changes: 19 additions & 16 deletions docs/docs/networks.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,15 @@ The chain identifier of our production network is fetchhub-4.
| Date range | 05/04/2022 --> |
| Denomination | afet |
| Decimals | 18 (1fet = 1000000000000000000afet) |
| Version | [v0.10.2](https://github.com/fetchai/fetchd/releases/tag/v0.10.2) |
| Version | [v0.10.3](https://github.com/fetchai/fetchd/releases/tag/v0.10.3) |
| RPC Endpoint | <https://rpc-fetchhub.fetch.ai:443> |
| GRPC Endpoint | <https://grpc-fetchhub.fetch.ai:443> |
| REST Endpoint | <https://rest-fetchhub.fetch.ai:443> |
| Block Explorer | [https://explore-fetchhub.fetch.ai](https://explore-fetchhub.fetch.ai) |
| Token Faucet | N/A |
| Genesis | `curl https://raw.githubusercontent.com/fetchai/genesis-fetchhub/fetchhub-4/fetchhub-4/data/genesis_migrated_5300200.json --output ~/.fetchd/config/genesis.json` |
| Genesis | `curl https://raw.githubusercontent.com/fetchai/genesis-fetchhub/fetchhub-4/fetchhub-4/data/genesis_migrated_5300200.json --output ~/.fetchd/config/genesis.json` |
| Seed Node(s) | 17693da418c15c95d629994a320e2c4f51a8069b@connect-fetchhub.fetch.ai:36456,a575c681c2861fe945f77cb3aba0357da294f1f2@connect-fetchhub.fetch.ai:36457,d7cda986c9f59ab9e05058a803c3d0300d15d8da@connect-fetchhub.fetch.ai:36458 |
| Snapshots | <https://storage.googleapis.com/fetch-ai-mainnet-snapshots/fetchhub-4-pruned.tgz> <br /> <https://storage.googleapis.com/fetch-ai-mainnet-snapshots/fetchhub-4-full.tgz> <br /> <https://storage.googleapis.com/fetch-ai-mainnet-snapshots/fetchhub-4-archive.tgz> |

## Mainnet Archives

Expand Down Expand Up @@ -99,17 +100,19 @@ This network is running the same software as our mainnet (`fetchhub-3`), and is

This network is used for testing the future upgrade to mainnet.

| Parameter | Value |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Chain ID | dorado-1 |
| Denomination | atestfet |
| Decimals | 18 (1testfet = 1000000000000000000atestfet) |
| Min Gas Prices | 1000000000atestfet |
| Version | [v0.10.x (fetchd >= v0.10.x)](https://github.com/fetchai/fetchd/releases/tag/v0.10.0) |
| RPC Endpoint | <https://rpc-dorado.fetch.ai:443> |
| GRPC Endpoint | <https://grpc-dorado.fetch.ai:443> |
| REST Endpoint | <https://rest-dorado.fetch.ai:443> |
| Block Explorer | [https://explore-dorado.fetch.ai/](https://explore-dorado.fetch.ai/) |
| Ledger Explorer | [https://browse-dorado.fetch.ai/](https://browse-dorado.fetch.ai/) |
| Token Faucet | Use block explorer |
| Seed Node(s) | [email protected]:36556,[email protected]:36557,[email protected]:36558 |
| Parameter | Value |
| --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Chain ID | dorado-1 |
| Denomination | atestfet |
| Decimals | 18 (1testfet = 1000000000000000000atestfet) |
| Min Gas Prices | 1000000000atestfet |
| Version | [v0.10.x (fetchd >= v0.10.3)](https://github.com/fetchai/fetchd/releases/tag/v0.10.0) |
| RPC Endpoint | <https://rpc-dorado.fetch.ai:443> |
| GRPC Endpoint | <https://grpc-dorado.fetch.ai:443> |
| REST Endpoint | <https://rest-dorado.fetch.ai:443> |
| Block Explorer | [https://explore-dorado.fetch.ai/](https://explore-dorado.fetch.ai/) |
| Ledger Explorer | [https://browse-dorado.fetch.ai/](https://browse-dorado.fetch.ai/) |
| Token Faucet | Use block explorer |
| Genesis | `curl https://storage.googleapis.com/fetch-ai-testnet-genesis/genesis-dorado-827201.json --output ~/.fetchd/config/genesis.json` |
| Seed Node(s) | [email protected]:36556,[email protected]:36557,[email protected]:36558 |
| Snapshots | <https://storage.googleapis.com/fetch-ai-testnet-snapshots/dorado-pruned.tgz> <br /> <https://storage.googleapis.com/fetch-ai-testnet-snapshots/dorado-full.tgz> <br /> <https://storage.googleapis.com/fetch-ai-testnet-snapshots/dorado-archive.tgz> |
52 changes: 52 additions & 0 deletions docs/docs/snapshots.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Chain State Snapshots

As blockchains get longer, the process of syncing from the genesis block begins to take many hours, or even days to complete.
In circumstances where a faster sync is required, various snapshots of the fetchd chain state data are available for download, to more quickly bootstrap a node.

Snapshots are available for both mainnet and the most recent testnet. The URLs can be obtained from the [network page](../networks/).
We aim to update snapshots on a daily basis.

The example below uses the pruned mainnet snapshot, but can be adapted as required for full or archive nodes.

## Using a snapshot

### Stop your node

If you are already running fetchd, it is important that you stop it before proceeding. Instructions for this are highly installation dependent and beyond the scope of this document, but could be as simple as a Ctrl-C.
If you have not already initialised your node, follow the instructions for [joining a testnet](../joining-a-testnet/) (modifying for mainnet as appropriate), then return to this page before starting fetchd.

### Reset your node

WARNING: This will irreversibly erase your node's state database. Ensure you take whatever backups you deem appropriate before proceeding.

If using fetchd <= 0.10.3
`fetchd unsafe-reset-all`

If using fetchd >= 0.10.4
`fetchd tendermint reset-state`

### Download and install the snapshot

Many options here! The example below assumes a bash-like environment, uses a single connection for downloading, confirms the md5sum of the downloaded data against that of the original, and does not land the original compressed data to disk. This is a good starting point, but depending on your local environment you may wish to make adaptations that eg sacrifice disk space and extra md5sum complexity for the benefit of parallel downloads with aria2. Entirely up to you... let us know how you get on!

```bash
# (optional) show the timestamp of the latest available snapshot
echo "Latest available snapshot timestamp : $(curl -s -I https://storage.googleapis.com/fetch-ai-mainnet-snapshots/fetchhub-4-pruned.tgz | grep last-modified | cut -f3- -d' ')"

# download, decompress and extract state database
curl -v https://storage.googleapis.com/fetch-ai-mainnet-snapshots/fetchhub-4-pruned.tgz -o- 2>headers.out | tee >(md5sum > md5sum.out) | gunzip -c | tar -xvf - --directory=~/.fetchd

# (optional, but recommended) compare source md5 checksum provided in the headers by google, with the one calculated locally
[[ $(grep 'x-goog-hash: md5' headers.out | sed -z 's/^.*md5=\(.*\)/\1/g' | tr -d '\r' | base64 -d | od -An -vtx1 | tr -d ' \n') == $(awk '{ print $1 }' md5sum.out) ]] && echo "OK - md5sum match" || echo "ERROR - md5sum MISMATCH"

# (optional) show the creation date of the downloaded snapshot
echo "Downloaded snapshot timestamp: $(grep last-modified headers.out | cut -f3- -d' ')"
```

### Restart your node

Again, this entirely depends on your local installation, but a simple example for mainnet might be...

```bash
fetchd start --p2p.seeds 17693da418c15c95d629994a320e2c4f51a8069b@connect-fetchhub.fetch.ai:36456,a575c681c2861fe945f77cb3aba0357da294f1f2@connect-fetchhub.fetch.ai:36457,d7cda986c9f59ab9e05058a803c3d0300d15d8da@connect-fetchhub.fetch.ai:36458`.
```
1 change: 1 addition & 0 deletions docs/mkdocs.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ nav:
- Installation: 'building.md'
- Versions: 'versions.md'
- Networks: 'networks.md'
- Snapshots: 'snapshots.md'
- Commandline Client:
- Introduction: 'cli-introduction.md'
- Keys: 'cli-keys.md'
Expand Down

0 comments on commit e7b7407

Please sign in to comment.