From 970d4cc64ad0313b523d4d69a3a80d14fa74f03d Mon Sep 17 00:00:00 2001 From: "mergify[bot]" <37929162+mergify[bot]@users.noreply.github.com> Date: Tue, 20 Jul 2021 12:30:42 +0200 Subject: [PATCH] docs: add chain upgrade guide v0.43 (backport #9567) (#9725) * docs: add chain upgrade guide v0.43 (#9567) ## Description This pull request adds a chain upgrade guide and example for v0.43. This pull request also removes migration documentation specific to v0.42 and adds links to v0.42 documentation when referencing genesis migrations. --- ### Author Checklist *All items are required. Please add a note to the item if the item is not applicable and please add links to any relevant follow up issues.* I have... - [x] included the correct `docs:` prefix in the PR title - [x] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting)) - [x] provided a link to the relevant issue or specification - n/a - [x] followed the [documentation writing guidelines](https://github.com/cosmos/cosmos-sdk/blob/master/docs/DOC_WRITING_GUIDELINES.md) - [x] reviewed "Files changed" and left comments if necessary - [x] confirmed all CI checks have passed ### Reviewers Checklist *All items are required. Please add a note if the item is not applicable and please add your handle next to the items reviewed if you only reviewed selected items.* I have... - [x] confirmed the correct `docs:` prefix in the PR title - [x] confirmed all author checklist items have been addressed - [ ] confirmed that this PR only changes documentation - [ ] reviewed content for consistency - [ ] reviewed content for thoroughness - [ ] reviewed content for spelling and grammar - [ ] tested instructions (if applicable) (cherry picked from commit 6098d7ed556e36d5dc67920248dcc462a4aaa2c0) # Conflicts: # cosmovisor/README.md * resolve conflicts Co-authored-by: Ryan Christoffersen <12519942+ryanchristo@users.noreply.github.com> Co-authored-by: ryanchrypto <12519942+ryanchrypto@users.noreply.github.com> --- cosmovisor/README.md | 14 +- docs/core/upgrade.md | 2 +- docs/migrations/chain-upgrade-guide-043.md | 237 +++++++++++++++++++++ docs/migrations/rest.md | 8 +- 4 files changed, 244 insertions(+), 17 deletions(-) create mode 100644 docs/migrations/chain-upgrade-guide-043.md diff --git a/cosmovisor/README.md b/cosmovisor/README.md index e263966a49a0..33becafcb9e6 100644 --- a/cosmovisor/README.md +++ b/cosmovisor/README.md @@ -39,7 +39,7 @@ All arguments passed to `cosmovisor` will be passed to the application binary (a └── $DAEMON_NAME ``` -The `cosmovisor/` directory incudes a subdirectory for each version of the application (i.e. `genesis` or `upgrades/`). Within each subdirectory is the application binary (i.e. `bin/$DAEMON_NAME`) and any additional auxiliary files associated with each binary. `current` is a symbolic link to the currently active directory (i.e. `genesis` or `upgrades/`). The `name` variable in `upgrades/` is the URI-encoded name of the upgrade as specified in the upgrade module plan. +The `cosmovisor/` directory incudes a subdirectory for each version of the application (i.e. `genesis` or `upgrades/`). Within each subdirectory is the application binary (i.e. `bin/$DAEMON_NAME`) and any additional auxiliary files associated with each binary. `current` is a symbolic link to the currently active directory (i.e `genesis` or `upgrades/`). The `name` variable in `upgrades/` is the URI-encoded name of the upgrade as specified in the upgrade module plan. Please note that `$DAEMON_HOME/cosmovisor` only stores the *application binaries*. The `cosmovisor` binary itself can be stored in any typical location (e.g. `/usr/local/bin`). The application will continue to store its data in the default data directory (e.g. `$HOME/.gaiad`) or the data directory specified with the `--home` flag. `$DAEMON_HOME` is independent of the data directory and can be set to any location. If you set `$DAEMON_HOME` to the same directory as the data directory, you will end up with a configuation like the following: @@ -90,7 +90,7 @@ https://example.com/testnet-1001-info.json?checksum=sha256:deaaa99fda9407c4dbe1d When `cosmovisor` is triggered to download the new binary, `cosmovisor` will parse the `"binaries"` field, download the new binary with [go-getter](https://github.com/hashicorp/go-getter), and unpack the new binary in the `upgrades/` folder so that it can be run as if it was installed manually. -Note that for this mechanism to provide strong security guarantees, all URLs should include a SHA 256/512 checksum. This ensures that no false binary is run, even if someone hacks the server or hijacks the DNS. `go-getter` will always ensure the downloaded file matches the checksum if it is provided. `go-getter` will also handle unpacking archives into directories (in this case the download link should point to a `zip` file of all data in the `bin` directory). +Note that for this mechanism to provide strong security guarantees, all URLs should include a SHA 256/512 checksum. This ensures that no false binary is run, even if someone hacks the server or hijacks the DNS. `go-getter` will always ensure the downloaded file matches the checksum if it is provided. To properly create a sha256 checksum on linux, you can use the `sha256sum` utility. For example: @@ -106,13 +106,7 @@ You can also use `sha512sum` if you would prefer to use longer hashes, or `md5su The following instructions provide a demonstration of `cosmovisor` using the simulation application (`simapp`) shipped with the Cosmos SDK's source code. The following commands are to be run from within the `cosmos-sdk` repository. -First, check out the latest `v0.42` release: - -``` -git checkout v0.42.7 -``` - -Compile the `simd` binary: +First compile the `simd` binary: ``` make build @@ -192,7 +186,7 @@ app.UpgradeKeeper = upgradekeeper.NewKeeper(skipUpgradeHeights, keys[upgradetype After that line, add the following: -```go + ```go app.UpgradeKeeper.SetUpgradeHandler("test1", func(ctx sdk.Context, plan upgradetypes.Plan) { // Add some coins to a random account addr, err := sdk.AccAddressFromBech32("cosmos18cgkqduwuh253twzmhedesw3l7v3fm37sppt58") diff --git a/docs/core/upgrade.md b/docs/core/upgrade.md index fbbba9e1bea3..717013b7c4cf 100644 --- a/docs/core/upgrade.md +++ b/docs/core/upgrade.md @@ -14,7 +14,7 @@ The Cosmos SDK uses two methods to perform upgrades. - Exporting the entire application state to a JSON file using the `export` CLI command, making changes, and then starting a new binary with the changed JSON file as the genesis file. See [Chain Upgrade Guide to v0.42](/v0.42/migrations/chain-upgrade-guide-040.html). -- Version v0.44 and later can perform upgrades in place to significantly decrease the upgrade time for chains with a larger state. Use the [Module Upgrade Guide](../building-modules/upgrade.md) to set up your application modules to take advantage of in-place upgrades. +- Version v0.43 and later can perform upgrades in place to significantly decrease the upgrade time for chains with a larger state. Use the [Module Upgrade Guide](../building-modules/upgrade.md) to set up your application modules to take advantage of in-place upgrades. This document provides steps to use the In-Place Store Migrations upgrade method. diff --git a/docs/migrations/chain-upgrade-guide-043.md b/docs/migrations/chain-upgrade-guide-043.md new file mode 100644 index 000000000000..129d22314c59 --- /dev/null +++ b/docs/migrations/chain-upgrade-guide-043.md @@ -0,0 +1,237 @@ + + +# Chain Upgrade Guide to v0.43 + +This document provides guidelines for a chain upgrade from v0.42 to v0.43 and an example of the upgrade process using `simapp`. {synopsis} + +::: tip +You must upgrade to Stargate v0.42 before upgrading to v0.43. If you have not done so, please see [Chain Upgrade Guide to v0.42](/v0.42/migrations/chain-upgrade-guide-040.html). +::: + +## Prerequisite Readings + +- [Upgrading Modules](../building-modules/upgrade.html) {prereq} +- [In-Place Store Migrations](../core/upgrade.html) {prereq} +- [Cosmovisor](../run-node/cosmovisor.html) {prereq} + +Cosmos SDK v0.43 introduces a new way of handling chain upgrades that no longer requires exporting state to JSON, making the necesssary changes, and then creating a new chain with the modified JSON as the new genesis file. + +Instead of starting a new chain, the upgrade binary will read the existing database and perform in-place store migrations. This new way of handling chain upgrades can be used alongside [Cosmovisor](../run-node/cosmovisor.html) to make the upgrade process seamless. + +## In-Place Store Migrations + +We recommend using [In-Place Store Migrations](../core/upgrade.html) to upgrade your chain from v0.42 to v0.43. The first step is to make sure all your modules follow the [Module Upgrade Guide](../building-modules/upgrade.html). The second step is to add an [upgrade handler](../core/upgrade.html#running-migrations) to `app.go`. + +In this document, we'll provide an example of what the upgrade handler looks like for a chain upgrading module versions for the first time. It's critical to note that the initial version of each module must be set to `1` rather than `0` or else the upgrade handler will re-initialize each module. + +In addition to migrating existing modules, the upgrade handler also performs store upgrades for new modules. In the example below, we'll be adding store migrations for two new modules made available in v0.43: `x/authz` and `x/feegrant`. + +## Using Cosmovisor + +We recommend validators use [Cosmovisor](../run-node/cosmovisor.html), which is a process manager for running application binaries. For security reasons, we recommend validators build their own upgrade binaries rather than enabling the auto-download option. Validators may still choose to use the auto-download option if the necessary security guarantees are in place (i.e. the URL provided in the upgrade proposal for the downloadable upgrade binary includes a proper checksum). + +Validators can use the auto-restart option to prevent unecessary downtime during the upgrade process. The auto-restart option will automatically restart the chain with the upgrade binary once the chain has halted at the proposed upgrade height. With the auto-restart option, validators can prepare the upgrade binary in advance and then relax at the time of the upgrade. + +## Migrating app.toml + +With the update to `v0.43`, new server configuration options have been added to `app.toml`. The updates include new configuration sections for Rosetta and gRPC Web as well as a new configuration option for State Sync. Check out the default [`app.toml`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.43.x/server/config/toml.go) file in the latest version of `v0.43` for more information. + +## Example: Simapp Upgrade + +The following example will walk through the upgrade process using `simapp` as our blockchain application. We will be upgrading `simapp` from v0.42 to v0.43. We will be building the upgrade binary ourselves and enabling the auto-restart option. + +*Note: In this example, we will be starting a new chain from `v0.42`. The binary for this version will be the genesis binary. For validators using Cosmovisor for the first time, the binary for the current version of the chain should be used as the genesis binary (i.e. the starting binary). For more information, see [Cosmovisor](../run-node/cosmovisor.html).* + +### Initial Setup + +From within the `cosmos-sdk` repository, check out the latest `v0.42.x` release: + +``` +git checkout release/v0.42.x +``` + +Build the `simd` binary for the latest `v0.42.x` release (the genesis binary): + +``` +make build +``` + +Reset `~/.simapp` (never do this in a production environment): + +``` +./build/simd unsafe-reset-all +``` + +Configure the `simd` binary for testing: + +``` +./build/simd config chain-id test +./build/simd config keyring-backend test +./build/simd config broadcast-mode block +``` + +Initialize the node and overwrite any previous genesis file (never do this in a production environment): + + + +``` +./build/simd init test --chain-id test --overwrite +``` + +Set the minimum gas price to `0stake` in `~/.simapp/config/app.toml`: + +``` +minimum-gas-prices = "0stake" +``` + +For the purpose of this demonstration, change `voting_period` in `genesis.json` to a reduced time of 20 seconds (`20s`): + +``` +cat <<< $(jq '.app_state.gov.voting_params.voting_period = "20s"' $HOME/.simapp/config/genesis.json) > $HOME/.simapp/config/genesis.json +``` + +Create a new key for the validator, then add a genesis account and transaction: + + + + +``` +./build/simd keys add validator +./build/simd add-genesis-account validator 5000000000stake --keyring-backend test +./build/simd gentx validator 1000000stake --chain-id test +./build/simd collect-gentxs +``` + +Now that our node is initialized and we are ready to start a new `simapp` chain, let's set up `cosmovisor` and the genesis binary. + +### Cosmovisor Setup + +First, install or update `cosmovisor`: + +``` +go get github.com/cosmos/cosmos-sdk/cosmovisor/cmd/cosmovisor +``` + +Set the required environment variables: + +``` +export DAEMON_NAME=simd +export DAEMON_HOME=$HOME/.simapp +``` + +Set the optional environment variable to trigger an automatic restart: + +``` +export DAEMON_RESTART_AFTER_UPGRADE=true +``` + +Create the folder for the genesis binary and copy the `v0.42.x` binary: + +``` +mkdir -p $DAEMON_HOME/cosmovisor/genesis/bin +cp ./build/simd $DAEMON_HOME/cosmovisor/genesis/bin +``` + +Now that `cosmovisor` is installed and the genesis binary has been added, let's add the upgrade handler to `simapp/app.go` and prepare the upgrade binary. + +### Chain Upgrade + + + +Check out `release/v0.43.x`: + +``` +git checkout release/v0.43.x +``` + +Add the following to `simapp/app.go` inside `NewSimApp` and after `app.UpgradeKeeper`: + +```go + app.registerUpgradeHandlers() +``` + +Add the following to `simapp/app.go` after `NewSimApp` (to learn more about the upgrade handler, see the [In-Place Store Migrations](../core/upgrade.html)): + +```go +func (app *SimApp) registerUpgradeHandlers() { + app.UpgradeKeeper.SetUpgradeHandler("v0.43", func(ctx sdk.Context, plan upgradetypes.Plan, _ module.VersionMap) (module.VersionMap, error) { + // 1st-time running in-store migrations, using 1 as fromVersion to + // avoid running InitGenesis. + fromVM := map[string]uint64{ + "auth": 1, + "bank": 1, + "capability": 1, + "crisis": 1, + "distribution": 1, + "evidence": 1, + "gov": 1, + "mint": 1, + "params": 1, + "slashing": 1, + "staking": 1, + "upgrade": 1, + "vesting": 1, + "ibc": 1, + "genutil": 1, + "transfer": 1, + } + + return app.mm.RunMigrations(ctx, app.configurator, fromVM) + }) + + upgradeInfo, err := app.UpgradeKeeper.ReadUpgradeInfoFromDisk() + if err != nil { + panic(err) + } + + if upgradeInfo.Name == "v0.43" && !app.UpgradeKeeper.IsSkipHeight(upgradeInfo.Height) { + storeUpgrades := storetypes.StoreUpgrades{ + Added: []string{"authz", "feegrant"}, + } + + // configure store loader that checks if version == upgradeHeight and applies store upgrades + app.SetStoreLoader(upgradetypes.UpgradeStoreLoader(upgradeInfo.Height, &storeUpgrades)) + } +} +``` + +Add `storetypes` to imports: + +```go + storetypes "github.com/cosmos/cosmos-sdk/store/types" +``` + +Build the `simd` binary for `v0.43.x` (the upgrade binary): + +``` +make build +``` + +Create the folder for the upgrade binary and copy the `v0.43.x` binary: + +``` +mkdir -p $DAEMON_HOME/cosmovisor/upgrades/v0.43/bin +cp ./build/simd $DAEMON_HOME/cosmovisor/upgrades/v0.43/bin +``` + +Now that we have added the upgrade handler and prepared the upgrade binary, we are ready to start `cosmovisor` and simulate the upgrade proposal process. + +### Upgrade Proposal + +Start the node using `cosmovisor`: + +``` +cosmovisor start +``` + +Open a new terminal window and submit an upgrade proposal along with a deposit and a vote (these commands must be run within 20 seconds of each other): + +``` +./build/simd tx gov submit-proposal software-upgrade v0.43 --title upgrade --description upgrade --upgrade-height 20 --from validator --yes +./build/simd tx gov deposit 1 10000000stake --from validator --yes +./build/simd tx gov vote 1 yes --from validator --yes +``` + +Confirm the chain automatically upgrades at height 20. diff --git a/docs/migrations/rest.md b/docs/migrations/rest.md index 6ed555613f84..9978974a2730 100644 --- a/docs/migrations/rest.md +++ b/docs/migrations/rest.md @@ -4,15 +4,11 @@ order: 2 # REST Endpoints Migration -Migrate to gRPC-Gateway REST endpoints. Legacy REST endpoints were marked as deprecated in v0.40 and will be removed in v0.45. {synopsis} - -::: warning -Two Legacy REST endpoints (`POST /txs` and `POST /txs/encode`) were removed ahead of schedule in v0.44 due to a security vulnerability. -::: +Migrate your REST endpoints. Legacy REST endpoints have been deprecated since v0.40 and they will be removed in v0.44. {synopsis} ## Deprecation of Legacy REST Endpoints -Cosmos SDK versions v0.39 and earlier registered REST endpoints using a package called `gorilla/mux`. These REST endpoints were marked as deprecated in v0.40 and have since been referred to as legacy REST endpoints. Legacy REST endpoints will be officially removed in v0.45. +The Cosmos SDK versions v0.39 and earlier provided REST endpoints to query the state and broadcast transactions. These endpoints were kept in Cosmos SDK v0.40 and they are still available in v0.43, but they are marked as deprecated and will be removed in v0.44. We therefore call these endpoints legacy REST endpoints. Some important information concerning all legacy REST endpoints: