From 6098d7ed556e36d5dc67920248dcc462a4aaa2c0 Mon Sep 17 00:00:00 2001 From: Ryan Christoffersen <12519942+ryanchristo@users.noreply.github.com> Date: Mon, 19 Jul 2021 08:35:56 -0700 Subject: [PATCH] docs: add chain upgrade guide v0.43 (#9567) ## Description This pull request adds a chain upgrade guide and example for v0.43. This pull request also removes migration documentation specific to v0.42 and adds links to v0.42 documentation when referencing genesis migrations. --- ### Author Checklist *All items are required. Please add a note to the item if the item is not applicable and please add links to any relevant follow up issues.* I have... - [x] included the correct `docs:` prefix in the PR title - [x] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting)) - [x] provided a link to the relevant issue or specification - n/a - [x] followed the [documentation writing guidelines](https://github.com/cosmos/cosmos-sdk/blob/master/docs/DOC_WRITING_GUIDELINES.md) - [x] reviewed "Files changed" and left comments if necessary - [x] confirmed all CI checks have passed ### Reviewers Checklist *All items are required. Please add a note if the item is not applicable and please add your handle next to the items reviewed if you only reviewed selected items.* I have... - [x] confirmed the correct `docs:` prefix in the PR title - [x] confirmed all author checklist items have been addressed - [ ] confirmed that this PR only changes documentation - [ ] reviewed content for consistency - [ ] reviewed content for thoroughness - [ ] reviewed content for spelling and grammar - [ ] tested instructions (if applicable) --- cosmovisor/README.md | 128 ++++++----- docs/building-modules/upgrade.md | 12 +- docs/core/upgrade.md | 16 +- docs/migrations/app_and_modules.md | 239 --------------------- docs/migrations/chain-upgrade-guide-040.md | 166 -------------- docs/migrations/chain-upgrade-guide-043.md | 237 ++++++++++++++++++++ docs/migrations/keyring.md | 33 --- docs/migrations/rest.md | 10 +- 8 files changed, 334 insertions(+), 507 deletions(-) delete mode 100644 docs/migrations/app_and_modules.md delete mode 100644 docs/migrations/chain-upgrade-guide-040.md create mode 100644 docs/migrations/chain-upgrade-guide-043.md delete mode 100644 docs/migrations/keyring.md diff --git a/cosmovisor/README.md b/cosmovisor/README.md index c32c39834582..33becafcb9e6 100644 --- a/cosmovisor/README.md +++ b/cosmovisor/README.md @@ -1,6 +1,8 @@ # Cosmosvisor Quick Start -`cosmovisor` is a small process manager for Cosmos SDK binaries that monitors the governance module via stdout for incoming chain upgrade proposals. If it sees a proposal that gets approved, it can be run manually or automatically to download the new binary, stop the current binary, run the migration script, replace the old node binary with the new one, and finally restart the node with the new genesis file. +`cosmovisor` is a small process manager for Cosmos SDK application binaries that monitors the governance module via stdout for incoming chain upgrade proposals. If it sees a proposal that gets approved, `cosmovisor` can automatically download the new binary, stop the current binary, switch from the old binary to the new one, and finally restart the node with the new binary. + +*Note: If new versions of the application are not set up to run in-place store migrations, migrations will need to be run manually before restarting `cosmovisor` with the new binary. For this reason, we recommend applications adopt in-place store migrations.* ## Installation @@ -12,16 +14,16 @@ go get github.com/cosmos/cosmos-sdk/cosmovisor/cmd/cosmovisor ## Command Line Arguments And Environment Variables -All arguments passed to the `cosmovisor` program will be passed to the current daemon binary (as a subprocess). `cosmovisor` will return `/dev/stdout` and `/dev/stderr` of the subprocess as its own. For this reason, `cosmovisor` cannot accept any command line arguments, nor print anything to output (unless it terminates unexpectedly before executing a binary). +All arguments passed to `cosmovisor` will be passed to the application binary (as a subprocess). `cosmovisor` will return `/dev/stdout` and `/dev/stderr` of the subprocess as its own. For this reason, `cosmovisor` cannot accept any command-line arguments other than those available to the application binary, nor will it print anything to output other than what is printed by the application binary. `cosmovisor` reads its configuration from environment variables: -* `DAEMON_HOME` is the location where the `cosmovisor/` directory is kept that contains the upgrade binaries (e.g. `$HOME/.gaiad`, `$HOME/.regend`, `$HOME/.simd`, etc.). +* `DAEMON_HOME` is the location where the `cosmovisor/` directory is kept that contains the genesis binary, the upgrade binaries, and any additional auxiliary files associated with each binary (e.g. `$HOME/.gaiad`, `$HOME/.regend`, `$HOME/.simd`, etc.). * `DAEMON_NAME` is the name of the binary itself (e.g. `gaiad`, `regend`, `simd`, etc.). * `DAEMON_ALLOW_DOWNLOAD_BINARIES` (*optional*), if set to `true`, will enable auto-downloading of new binaries (for security reasons, this is intended for full nodes rather than validators). By default, `cosmovisor` will not auto-download new binaries. -* `DAEMON_RESTART_AFTER_UPGRADE` (*optional*), if set to `true`, will restart the subprocess with the same command line arguments and flags (but with the new binary) after a successful upgrade. By default, `cosmovisor` stops running after an upgrade and requires the system administrator to manually restart it. Note that `cosmovisor` will not auto-restart the subprocess if there was an error. +* `DAEMON_RESTART_AFTER_UPGRADE` (*optional*), if set to `true`, will restart the subprocess with the same command-line arguments and flags (but with the new binary) after a successful upgrade. By default, `cosmovisor` stops running after an upgrade and requires the system administrator to manually restart it. Note that `cosmovisor` will not auto-restart the subprocess if there was an error. -## Data Folder Layout +## Folder Layout `$DAEMON_HOME/cosmovisor` is expected to belong completely to `cosmovisor` and the subprocesses that are controlled by it. The folder content is organized as follows: @@ -37,11 +39,9 @@ All arguments passed to the `cosmovisor` program will be passed to the current d └── $DAEMON_NAME ``` -Each version of the Cosmos SDK application is stored under either `genesis` or `upgrades/`, which holds `bin/$DAEMON_NAME` along with any other needed files such as auxiliary client programs or libraries. `current` is a symbolic link to the currently active folder and `current/bin/$DAEMON_NAME` is the currently active binary. - -*Note: The `name` variable in `upgrades/` holds the URI-encoded name of the upgrade as specified in the upgrade module plan.* +The `cosmovisor/` directory incudes a subdirectory for each version of the application (i.e. `genesis` or `upgrades/`). Within each subdirectory is the application binary (i.e. `bin/$DAEMON_NAME`) and any additional auxiliary files associated with each binary. `current` is a symbolic link to the currently active directory (i.e `genesis` or `upgrades/`). The `name` variable in `upgrades/` is the URI-encoded name of the upgrade as specified in the upgrade module plan. -Please note that `$DAEMON_HOME/cosmovisor` just stores the *binaries* and associated *program code*. The `cosmovisor` binary can be stored in any typical location (e.g. `/usr/local/bin`). The actual blockchain program will store its data under the default data directory (e.g. `$HOME/.gaiad`) which is independent of `$DAEMON_HOME`. `$DAEMON_HOME` can be set to any location. If you set `$DAEMON_HOME` to the default data directory, you will end up with a configuation like the following: +Please note that `$DAEMON_HOME/cosmovisor` only stores the *application binaries*. The `cosmovisor` binary itself can be stored in any typical location (e.g. `/usr/local/bin`). The application will continue to store its data in the default data directory (e.g. `$HOME/.gaiad`) or the data directory specified with the `--home` flag. `$DAEMON_HOME` is independent of the data directory and can be set to any location. If you set `$DAEMON_HOME` to the same directory as the data directory, you will end up with a configuation like the following: ``` .gaiad @@ -57,20 +57,20 @@ The system administrator is responsible for: - installing the `cosmovisor` binary - configuring the host's init system (e.g. `systemd`, `launchd`, etc.) - appropriately setting the environmental variables -- installing the `genesis` folder manually -- installing the `upgrades/` folders manually +- manually installing the `genesis` folder +- manually installing the `upgrades/` folders -`cosmovisor` will set the `current` link to point to `genesis` at first start (when no `current` link exists) and will handle switching binaries at the correct points in time so that the system administrator can prepare days in advance and relax at upgrade time. +`cosmovisor` will set the `current` link to point to `genesis` at first start (i.e. when no `current` link exists) and then handle switching binaries at the correct points in time so that the system administrator can prepare days in advance and relax at upgrade time. -Note that blockchain applications that wish to support upgrades may package up a genesis `cosmovisor` tarball with this information, just as they prepare the genesis binary tarball. In fact, they may package up a tarball with all upgrades up to a current point so that the upgrades can be easily downloaded for others who wish to sync a fullnode from start. +In order to support downloadable binaries, a tarball for each upgrade binary will need to be packaged up and made available through a canonical URL. Additionally, a tarball that includes the genesis binary and all available upgrade binaries can be packaged up and made available so that all the necessary binaries required to sync a fullnode from start can be easily downloaded. The `DAEMON` specific code and operations (e.g. tendermint config, the application db, syncing blocks, etc.) all work as expected. The application binaries' directives such as command-line flags and environment variables also work as expected. ## Auto-Download -Generally, the system requires that the system administrator place all relevant binaries on the disk before the upgrade happens. However, for people who don't need such control and want an easier setup (maybe they are syncing a non-validating fullnode and want to do little maintenance), there is another option. +Generally, `cosmovisor` requires that the system administrator place all relevant binaries on disk before the upgrade happens. However, for people who don't need such control and want an easier setup (maybe they are syncing a non-validating fullnode and want to do little maintenance), there is another option. -If you set `DAEMON_ALLOW_DOWNLOAD_BINARIES=true`, and no local binary can be found when an upgrade is triggered, `cosmovisor` will attempt to download and install the binary itself. The plan stored in the upgrade module has an info field for arbitrary JSON. This info is expected to be outputed on the halt log message. There are two valid formats to specify a download in such a message: +If `DAEMON_ALLOW_DOWNLOAD_BINARIES` is set to `true`, and no local binary can be found when an upgrade is triggered, `cosmovisor` will attempt to download and install the binary itself. The plan stored in the upgrade module has an info field for arbitrary JSON. This info is expected to be outputed on the halt log message. There are two valid formats to specify a download in such a message: 1. Store an os/architecture -> binary URI map in the upgrade plan info field as JSON under the `"binaries"` key. For example: @@ -88,11 +88,9 @@ If you set `DAEMON_ALLOW_DOWNLOAD_BINARIES=true`, and no local binary can be fou https://example.com/testnet-1001-info.json?checksum=sha256:deaaa99fda9407c4dbe1d04bd49bab0cc3c1dd76fa392cd55a9425be074af01e ``` -This file contained in the link will be retrieved by [go-getter](https://github.com/hashicorp/go-getter) and the `"binaries"` field will be parsed as above. - -If there is no local binary, `DAEMON_ALLOW_DOWNLOAD_BINARIES=true`, and we can access a canonical url for the new binary, then the `cosmovisor` will download it with [go-getter](https://github.com/hashicorp/go-getter) and unpack it into the `upgrades/` folder to be run as if we installed it manually. +When `cosmovisor` is triggered to download the new binary, `cosmovisor` will parse the `"binaries"` field, download the new binary with [go-getter](https://github.com/hashicorp/go-getter), and unpack the new binary in the `upgrades/` folder so that it can be run as if it was installed manually. -Note that for this mechanism to provide strong security guarantees, all URLs should include a SHA 256/512 checksum. This ensures that no false binary is run, even if someone hacks the server or hijacks the DNS. `go-getter` will always ensure the downloaded file matches the checksum if it is provided. `go-getter` will also handle unpacking archives into directories (in this case the download link should point to a `zip` file of all data in the `bin` directory). +Note that for this mechanism to provide strong security guarantees, all URLs should include a SHA 256/512 checksum. This ensures that no false binary is run, even if someone hacks the server or hijacks the DNS. `go-getter` will always ensure the downloaded file matches the checksum if it is provided. To properly create a sha256 checksum on linux, you can use the `sha256sum` utility. For example: @@ -104,80 +102,91 @@ The result will look something like the following: `29139e1381b8177aec909fab9a75 You can also use `sha512sum` if you would prefer to use longer hashes, or `md5sum` if you would prefer to use broken hashes. Whichever you choose, make sure to set the hash algorithm properly in the checksum argument to the URL. -## Example: simd +## Example: SimApp Upgrade -The following instructions provide a demonstration of `cosmovisor`'s integration with the `simd` application shipped along the Cosmos SDK's source code. The following commands are to be run from within the `cosmos-sdk` repository. +The following instructions provide a demonstration of `cosmovisor` using the simulation application (`simapp`) shipped with the Cosmos SDK's source code. The following commands are to be run from within the `cosmos-sdk` repository. -First compile `simd`: +First compile the `simd` binary: ``` make build ``` -Create a new key and set up the `simd` node: +Reset `~/.simapp` (never do this in a production environment): ``` -rm -rf $HOME/.simapp -./build/simd keys --keyring-backend=test add validator -./build/simd init testing --chain-id test -./build/simd add-genesis-account --keyring-backend=test validator 1000000000stake -./build/simd gentx --keyring-backend test --chain-id test validator 1000000stake -./build/simd collect-gentxs +./build/simd unsafe-reset-all ``` -Set the required environment variables: +Configure the `simd` binary for testing: ``` -export DAEMON_NAME=simd -export DAEMON_HOME=$HOME/.simapp +./build/simd config chain-id test +./build/simd config keyring-backend test +./build/simd config broadcast-mode block ``` -Set the optional environment variable to trigger an automatic restart: +Initialize the node and overwrite any previous genesis file (never do this in a production environment): + + ``` -export DAEMON_RESTART_AFTER_UPGRADE=true +./build/simd init test --chain-id test --overwrite ``` -Create the `cosmovisor`’s genesis folders and copy the `simd` binary: +Set the minimum gas price to `0stake` in `~/.simapp/config/app.toml`: ``` -mkdir -p $DAEMON_HOME/cosmovisor/genesis/bin -cp ./build/simd $DAEMON_HOME/cosmovisor/genesis/bin +minimum-gas-prices = "0stake" ``` -For the sake of this demonstration, amend `voting_params.voting_period` in `$HOME/.simapp/config/genesis.json` to a reduced time of ~5 minutes (`300s`) and then start `cosmosvisor`: +Create a new key for the validator, then add a genesis account and transaction: + + + ``` -cosmovisor start +./build/simd keys add validator +./build/simd add-genesis-account validator 1000000000stake --keyring-backend test +./build/simd gentx validator 1000000stake --chain-id test +./build/simd collect-gentxs ``` -Open a new terminal window and submit a software upgrade proposal: +Set the required environment variables: ``` -./build/simd tx gov submit-proposal software-upgrade test1 --title "upgrade-demo" --description "upgrade" --from validator --upgrade-height 100 --deposit 10000000stake --chain-id test --keyring-backend test -y +export DAEMON_NAME=simd +export DAEMON_HOME=$HOME/.simapp ``` -Query the proposal to ensure it was correctly broadcast and added to a block: +Set the optional environment variable to trigger an automatic restart: ``` -./build/simd query gov proposal 1 +export DAEMON_RESTART_AFTER_UPGRADE=true ``` -Submit a `yes` vote for the upgrade proposal: +Create the folder for the genesis binary and copy the `simd` binary: ``` -./build/simd tx gov vote 1 yes --from validator --keyring-backend test --chain-id test -y +mkdir -p $DAEMON_HOME/cosmovisor/genesis/bin +cp ./build/simd $DAEMON_HOME/cosmovisor/genesis/bin ``` -For the sake of this demonstration, we will hardcode a modification in `simapp` to simulate a code change. In `simapp/app.go`, find the line containing the `UpgradeKeeper` initialization. It should look like the following: +For the sake of this demonstration, amend `voting_period` in `genesis.json` to a reduced time of 20 seconds (`20s`): ``` +cat <<< $(jq '.app_state.gov.voting_params.voting_period = "20s"' $HOME/.simapp/config/genesis.json) > $HOME/.simapp/config/genesis.json +``` + +Next, we will hardcode a modification in `simapp` to simulate a code change. In `simapp/app.go`, find the line containing the `UpgradeKeeper` initialization. It should look like the following: + +```go app.UpgradeKeeper = upgradekeeper.NewKeeper(skipUpgradeHeights, keys[upgradetypes.StoreKey], appCodec, homePath) ``` -After that line, add the following snippet: +After that line, add the following: - ``` + ```go app.UpgradeKeeper.SetUpgradeHandler("test1", func(ctx sdk.Context, plan upgradetypes.Plan) { // Add some coins to a random account addr, err := sdk.AccAddressFromBech32("cosmos18cgkqduwuh253twzmhedesw3l7v3fm37sppt58") @@ -191,12 +200,31 @@ app.UpgradeKeeper.SetUpgradeHandler("test1", func(ctx sdk.Context, plan upgradet }) ``` -Now recompile a new binary and make a copy of it in `$DAEMON_HOME/cosmosvisor/upgrades/test1/bin` (you may need to run `export DAEMON_HOME=$HOME/.simapp` again if you are using a new window): +Now recompile the `simd` binary with the added upgrade handler: ``` make build +``` + +Create the folder for the upgrade binary and copy the `simd` binary: + +``` mkdir -p $DAEMON_HOME/cosmovisor/upgrades/test1/bin cp ./build/simd $DAEMON_HOME/cosmovisor/upgrades/test1/bin ``` -The upgrade will occur automatically at height 100. +Start `cosmosvisor`: + +``` +cosmovisor start +``` + +Open a new terminal window and submit an upgrade proposal along with a deposit and a vote (these commands must be run within 20 seconds of each other): + +``` +./build/simd tx gov submit-proposal software-upgrade test1 --title upgrade --description upgrade --upgrade-height 20 --from validator --yes +./build/simd tx gov deposit 1 10000000stake --from validator --yes +./build/simd tx gov vote 1 yes --from validator --yes +``` + +The upgrade will occur automatically at height 20. diff --git a/docs/building-modules/upgrade.md b/docs/building-modules/upgrade.md index 2f9eaa98f339..f1c37930cffa 100644 --- a/docs/building-modules/upgrade.md +++ b/docs/building-modules/upgrade.md @@ -2,9 +2,9 @@ order: 13 --> -# In-Place Store Migrations +# Upgrading Modules -In-place store migrations allow your modules to upgrade to new versions that include breaking changes. This document outlines how to build modules to take advantage of this functionality. {synopsis} +[In-Place Store Migrations](../core/upgrade.html) allow your modules to upgrade to new versions that include breaking changes. This document outlines how to build modules to take advantage of this functionality. {synopsis} ## Prerequisite Readings @@ -12,16 +12,16 @@ In-place store migrations allow your modules to upgrade to new versions that inc ## Consensus Version -Successful upgrades of existing modules require your `AppModule` to implement the function `ConsensusVersion() uint64`. +Successful upgrades of existing modules require each `AppModule` to implement the function `ConsensusVersion() uint64`. - The versions must be hard-coded by the module developer. - The initial version **must** be set to 1. -Consensus versions serve as state-breaking versions of app modules and are incremented when the module is upgraded. +Consensus versions serve as state-breaking versions of app modules and must be incremented when the module introduces breaking changes. ## Registering Migrations -To register the functionality that takes place during a module upgrade, you must register which migrations we want to take place. +To register the functionality that takes place during a module upgrade, you must register which migrations you want to take place. Migration registration takes place in the `Configurator` using the `RegisterMigration` method. The `AppModule` reference to the configurator is in the `RegisterServices` method. @@ -54,4 +54,4 @@ func (m Migrator) Migrate1to2(ctx sdk.Context) error { } ``` -To see example code of changes that were implemented in a migration of balance keys, check out the [func migrateBalanceKeys](https://github.com/cosmos/cosmos-sdk/blob/36f68eb9e041e20a5bb47e216ac5eb8b91f95471/x/bank/legacy/v043/store.go#L41-L62) code. For context, this code introduced migrations of the bank store that updated addresses to be prefixed by their length in bytes as outlined in [ADR-028](../architecture/adr-028-public-key-addresses.md). +To see example code of changes that were implemented in a migration of balance keys, check out [migrateBalanceKeys](https://github.com/cosmos/cosmos-sdk/blob/36f68eb9e041e20a5bb47e216ac5eb8b91f95471/x/bank/legacy/v043/store.go#L41-L62). For context, this code introduced migrations of the bank store that updated addresses to be prefixed by their length in bytes as outlined in [ADR-028](../architecture/adr-028-public-key-addresses.md). diff --git a/docs/core/upgrade.md b/docs/core/upgrade.md index c37d813871c7..717013b7c4cf 100644 --- a/docs/core/upgrade.md +++ b/docs/core/upgrade.md @@ -8,13 +8,13 @@ order: 15 Read and understand all of the in-place store migration documentation before you run a migration on a live chain. ::: -Upgrade your app modules smoothly with custom in-place migration logic. {synopsis} +Upgrade your app modules smoothly with custom in-place store migration logic. {synopsis} The Cosmos SDK uses two methods to perform upgrades. -- Exporting the entire application state to a JSON file using the `export` CLI command, making changes, and then starting a new binary with the changed JSON file as the genesis file. See the [Chain Upgrade Guide](../migrations/chain-upgrade-guide-040.md#upgrade-procedure). +- Exporting the entire application state to a JSON file using the `export` CLI command, making changes, and then starting a new binary with the changed JSON file as the genesis file. See [Chain Upgrade Guide to v0.42](/v0.42/migrations/chain-upgrade-guide-040.html). -- Version v0.43 and later can perform upgrades in place to significantly decrease the upgrade time for chains with a larger state. Use the [Migration Upgrade Guide](../building-modules/upgrade.md) guide to set up your application modules to take advantage of in-place upgrades. +- Version v0.43 and later can perform upgrades in place to significantly decrease the upgrade time for chains with a larger state. Use the [Module Upgrade Guide](../building-modules/upgrade.md) to set up your application modules to take advantage of in-place upgrades. This document provides steps to use the In-Place Store Migrations upgrade method. @@ -48,13 +48,13 @@ The version map is a mapping of module names to consensus versions. The map is p Upgrades use an `UpgradeHandler` to facilitate migrations. The `UpgradeHandler` functions implemented by the app developer must conform to the following function signature. These functions retrieve the `VersionMap` from x/upgrade's state and return the new `VersionMap` to be stored in x/upgrade after the upgrade. The diff between the two `VersionMap`s determines which modules need upgrading. -```golang +```go type UpgradeHandler func(ctx sdk.Context, plan Plan, fromVM VersionMap) (VersionMap, error) ``` Inside these functions, you must perform any upgrade logic to include in the provided `plan`. All upgrade handler functions must end with the following line of code: -```golang +```go return app.mm.RunMigrations(ctx, cfg, fromVM) ``` @@ -62,7 +62,7 @@ Inside these functions, you must perform any upgrade logic to include in the pro Migrations are run inside of an `UpgradeHandler` using `app.mm.RunMigrations(ctx, cfg, vm)`. The `UpgradeHandler` functions describe the functionality to occur during an upgrade. The `RunMigration` function loops through the `VersionMap` argument and runs the migration scripts for all versions that are less than the versions of the new binary app module. After the migrations are finished, a new `VersionMap` is returned to persist the upgraded module versions to state. -```golang +```go cfg := module.NewConfigurator(...) app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, vm module.VersionMap) (module.VersionMap, error) { @@ -76,7 +76,7 @@ app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgrad }) ``` -To learn more about configuring migration scripts for your modules, see the [Migration Upgrade Guide](../building-modules/upgrade.md). +To learn more about configuring migration scripts for your modules, see the [Module Upgrade Guide](../building-modules/upgrade.md). ## Adding New Modules During Upgrades @@ -86,7 +86,7 @@ You can introduce entirely new modules to the application during an upgrade. New All chains preparing to run in-place store migrations will need to manually add store upgrades for new modules and then configure the store loader to apply those upgrades. This ensures that the new module's stores are added to the multistore before the migrations begin. -```golang +```go upgradeInfo, err := app.UpgradeKeeper.ReadUpgradeInfoFromDisk() if err != nil { panic(err) diff --git a/docs/migrations/app_and_modules.md b/docs/migrations/app_and_modules.md deleted file mode 100644 index 7c93a1f6d5b2..000000000000 --- a/docs/migrations/app_and_modules.md +++ /dev/null @@ -1,239 +0,0 @@ - - -# App and Modules Migration - -The following document describes the changes to update your app and modules from Cosmos SDK v0.39 to v0.40, -a.k.a. Stargate release. {synopsis} - -## Update Tooling - -Make sure to have the following dependencies before updating your app to v0.40: - -- Go 1.15+ -- Docker -- Node.js v12.0+ (optional, for generating Swagger docs) - -In Cosmos-SDK we manage the project using Makefile. Your own app can use a similar Makefile to the [Cosmos SDK's one](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/Makefile). More specifically, below are some _make_ commands that might be useful for your own app, related to the introduction of Protocol Buffers: - -- `proto-update-deps` - To download/update the required thirdparty `proto` definitions. -- `proto-gen` - To auto generate proto code. -- `proto-check-breaking` - To check proto breaking changes. -- `proto-format` - To format proto files. - -## Updating Modules - -This section outlines how to upgrade your module to v0.40. There is also a whole section about [building modules](../building-modules/README.md) from scratch, it might serve as a useful guide. - -### Protocol Buffers - -As outlined in our [encoding guide](../core/encoding.md), one of the most significant improvements introduced in Cosmos SDK v0.40 is Protobuf. - -The rule of thumb is that any object that needs to be serialized (into binary or JSON) must implement `proto.Message` and must be serializable into Protobuf format. The easiest way to do it is to use Protobuf type definition and `protoc` compiler to generate the structures and functions for you. In practice, the three following categories of types must be converted to Protobuf messages: - -- client-facing types: `Msg`s, query requests and responses. This is because client will send these types over the wire to your app. -- objects that are stored in state. This is because the SDK stores the binary representation of these types in state. -- genesis types. These are used when importing and exporting state snapshots during chain upgrades. - -Let's have a look at [x/auth's](../../x/auth/spec/README.md) `BaseAccount` objects, which are stored in a state. The migration looks like: - -```diff -// We were definining `MsgSend` as a Go struct in v0.39. -- // https://github.com/cosmos/cosmos-sdk/blob/v0.39.2/x/bank/internal/types/msgs.go#L12-L16 -- type BaseAccount struct { -- Address sdk.AccAddress `json:"address" yaml:"address"` -- Coins sdk.Coins `json:"coins" yaml:"coins"` -- PubKey crypto.PubKey `json:"public_key" yaml:"public_key"` -- AccountNumber uint64 `json:"account_number" yaml:"account_number"` -- Sequence uint64 `json:"sequence" yaml:"sequence"` -- } - -// And it should be converted to a Protobuf message in v0.40. -+ // https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/proto/cosmos/auth/v1beta1/auth.proto#L13-L25 -+ message BaseAccount { -+ string address = 1; -+ google.protobuf.Any pub_key = 2 -+ [(gogoproto.jsontag) = "public_key,omitempty", (gogoproto.moretags) = "yaml:\"public_key\""]; -+ uint64 account_number = 3 [(gogoproto.moretags) = "yaml:\"account_number\""]; -+ uint64 sequence = 4; -+ } -} -``` - -In general, we recommend to put all the Protobuf definitions in your module's subdirectory under a root `proto/` folder, as described in [ADR-023](../architecture/adr-023-protobuf-naming.md). This ADR also contains other useful information on naming conventions. - -You might have noticed that the `PubKey` interface in v0.39's `BaseAccount` has been transformed into an `Any`. For storing interfaces, we use Protobuf's `Any` message, which is a struct that can hold arbitrary content. Please refer to the [encoding FAQ](../core/encoding.md#faq) to learn how to handle interfaces and `Any`s. - -Once all your Protobuf messages are defined, use the `make proto-gen` command defined in the [tooling section](#tooling) to generate Go structs. These structs will be generated into `*.pb.go` files. As a quick example, here is the generated Go struct for the Protobuf BaseAccount we defined above: - -+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/x/auth/types/auth.pb.go#L28-L36 - -There might be some back and forth removing old Go structs/interfaces and defining new Protobuf messages before your Go app compiles and your tests pass. - -### Create `Msg` and `Query` Services - -Cosmos SDK v0.40 uses Protobuf services to define state transitions (`Msg`s) and state queries, please read [the building modules guide on those services](../building-modules/messages-and-queries.md) for an overview. - -#### `Msg` Service - -The handler pattern (inside the `handler.go` file) for handling `Msg`s is deprecated. You may still keep it if you wish to support `Msg`s defined in pre-Stargate versions of the SDK. However, it is strongly recommended to add a `Msg` service to your Protobuf files, and each old `Msg` handler should be converted into a service method. Taking [x/bank's](../../x/bank/spec/README.md) `MsgSend` as an example, we have a corresponding `/cosmos.bank.v1beta1.MsgSend` RPC: - -+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/proto/cosmos/bank/v1beta1/tx.proto#L10-L31 - -A state transition is therefore modelized as a Protobuf service method, with a method request, and an (optionally empty) method response. - -After defining your `Msg` service, run the `make proto-gen` script again to generate `Msg` server interfaces. The name of this interface is simply `MsgServer`. The implementation of this interface should follow exactly the implementation of the old `Msg` handlers, which, in most cases, defers the actual state transition logic to the [keeper](../building-modules/keeper.md). You may implement a `MsgServer` directly on the keeper, or you can do it using a new struct (e.g. called `msgServer`) that references the module's keeper. - -For more information, please check our [`Msg` service guide](../building-modules/msg-services.md). - -#### `Query` Service - -The querier pattern (inside the `querier.go` file) for handling state queries is deprecated. You may still keep this file to support legacy queries, but it is strongly recommended to use a Protobuf `Query` service to handle state queries. - -Each query endpoint is now defined as a separate service method in the `Query` service. Still taking `x/bank` as an example, here are the queries to fetch an account's balances: - -+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/proto/cosmos/bank/v1beta1/query.proto#L12-L23 - -Each query has its own `Request` and `Response` types. Please also note the `google.api.http` option (coming from [`grpc-gateway`](https://github.com/grpc-ecosystem/grpc-gateway)) on each service method. `grpc-gateway` is a tool that exposes `Query` service methods as REST endpoints. Adding this annotation will expose these endpoints not only as gRPC endpoints, but also as REST endpoints. An overview of gRPC versus REST can be found [here](../core/grpc_rest.md). - -After defining the `Query` Protobuf service, run the `make proto-gen` command to generate corresponding interfaces. The interface that needs to be implemented by your module is `QueryServer`. This interface can be implemented on the [keeper](../building-modules/keeper.md) directly, or on a struct (e.g. called `queryServer`) that references the module's keeper. The logic of the implementation, namely the logic that fetches data from the module's store and performs unmarshalling, can be deferred to the keeper. - -Cosmos SDK v0.40 also comes with an efficient pagination, it now uses `Prefix` stores to make queries. There are 2 helpers for pagination, [`Paginate`](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/types/query/pagination.go#L40-L42), [`FilteredPaginate`](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/types/query/filtered_pagination.go#L9-L17). - -For more information, please check our [`Query` service guide](../building-modules/query-services.md). - -#### Wiring up `Msg` and `Query` Services - -We added a new `RegisterServices` method that registers a module's `Msg` service and a `Query` service. It should be implemented by all modules, using a `Configurator` object: - -+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/x/bank/module.go#L99-L103 - -If you wish to expose your `Query` endpoints as REST endpoints (as proposed in the [`Query` Services paragraph](#query-services)), make sure to also implement the `RegisterGRPCGatewayRoutes` method: - -+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/x/bank/module.go#L69-L72 - -### Codec - -If you still use Amino (which is deprecated since Stargate), you must register related types using the `RegisterLegacyAminoCodec(cdc *codec.LegacyAmino)` method (previously it was called `RegisterCodec(cdc *codec.Codec)`). - -Moreover, a new `RegisterInterfaces` method has been added to the `AppModule` interface that all modules must implement. This method must register the interfaces that Protobuf messages implement, as well as the service request `Msg`s used in the module. An example from x/bank is given below: - -+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/x/bank/types/codec.go#L21-L34 - -### Keeper - -The `Keeper` constructor now takes a `codec.Codec` instead of a concrete Amino codec. This `codec.Codec` is used to encode types as binary and save the bytes into the state. With an interface, you can define `codec.Codec` to be Amino or Protobuf on an app level, and keepers will use that encoding library to encode state. Please note that Amino is deprecated in v0.40, and we strongly recommend to use Protobuf. Internally, the SDK still uses Amino in a couple of places (legacy REST API, x/params, keyring...), but Amino is planned to be removed in a future release. - -Keeping Amino for now can be useful if you wish to update to SDK v0.40 without doing a chain upgrade with a genesis migration. This will not require updating the stores, so you can migrate to Cosmos SDK v0.40 with less effort. However, as Amino will be removed in a future release, the chain migration with genesis export/import will need to be performed then. - -Related to the keepers, each module's `AppModuleBasic` now also includes this `codec.Codec`: - -+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/x/bank/module.go#L35-L38 - -### CLI - -Each modules may optionally expose CLI commands, and some changes are needed in these commands. - -First, `context.CLIContext` is renamed to `client.Context` and moved to `github.com/cosmos/cosmos-sdk/client`. - -Second, the global `viper` usage is removed from client and is replaced with Cobra' `cmd.Flags()`. There are two helpers to read common flags for CLI txs and queries: - -```go -clientCtx, err := client.GetClientQueryContext(cmd) -clientCtx, err := client.GetClientTxContext(cmd) -``` - -Some other flags helper functions are transformed: `flags.PostCommands(cmds ...*cobra.Command) []*cobra.Command` and `flags.GetCommands(...)` usage is now replaced by `flags.AddTxFlagsToCmd(cmd *cobra.Command)` and `flags.AddQueryFlagsToCmd(cmd *cobra.Command)` respectively. - -Moreover, new CLI commands don't take any codec as input anymore. Instead, the `clientCtx` can be retrieved from the `cmd` itself using the `GetClient{Query,Tx}Context` function above, and the codec as `clientCtx.Codec`. - -```diff -// v0.39 -- func SendTxCmd(cdc *codec.Codec) *cobra.Command { -- cdc.MarshalJSON(...) -- } - -// v0.43 -+ func NewSendTxCmd() *cobra.Command { -+ clientCtx, err := client.GetClientTxContext(cmd) -+ clientCtx.Codec.MarshalJSON(...) -+} -``` - -Finally, once your [`Query` services](#query-service) are wired up, the CLI commands should preferably use gRPC to communicate with the node. The gist is to create a `Query` or `Msg` client using the command's `clientCtx`, and perform the request using Protobuf's generated code. An example for querying x/bank balances is given here: - -+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/x/bank/client/cli/query.go#L66-L94 - -### Miscelleanous - -A number of other smaller breaking changes are also noteworthy. - -| Before | After | Comment | -| ----------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `alias.go` file | Removed | `alias` usage is removed, please see [#6311](https://github.com/cosmos/cosmos-sdk/issues/6311) for details. | -| `codec.New()` | `codec.NewLegacyAmino()` | Simple rename. | -| `DefaultGenesis()` | `DefaultGenesis(cdc codec.JSONCodec)` | `DefaultGenesis` takes a codec argument now | -| `ValidateGenesis()` | `ValidateGenesis(cdc codec.JSONCodec, config client.TxEncodingConfig, bz json.RawMessage)` | `ValidateGenesis` now requires `Marshaler`, `TxEncodingConfig`, `json.RawMessage` as input. | -| `Route() string` | `Route() sdk.Route` | For legacy handlers, return type of `Route()` method is changed from `string` to `"github.com/cosmos/cosmos-sdk/types".Route`. It should return a `NewRoute()` which includes `RouterKey` and `NewHandler` as params. | -| `QuerierHandler` | `LegacyQuerierHandler` | Simple rename. | -| `InitGenesis(ctx sdk.Context, data json.RawMessage) []abci.ValidatorUpdate` | InitGenesis(ctx sdk.Context, cdc codec.JSONCodec, data json.RawMessage) []abci.ValidatorUpdate | `InitGenesis` now takes a codec input. | -| `ExportGenesis(ctx sdk.Context, data json.RawMessage) []abci.ValidatorUpdate` | ExportGenesis(ctx sdk.Context, cdc codec.JSONCodec, data json.RawMessage) []abci.ValidatorUpdate | `ExportGenesis` now takes a codec input. | - -## Updating Your App - -For a reference implementation used for demo purposes, you can refer to the SDK's [SimApp](https://github.com/cosmos/cosmos-sdk/tree/v0.40.0-rc6/simapp) for your app's migration. The most important changes are described in this section. - -### Creating Codecs - -With the introduction of Protobuf, each app needs to define the encoding library (Amino or Protobuf) to be used throughout the app. There is a central struct, [`EncodingConfig`](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/simapp/params/encoding.go#L9-L11), which defines all information necessary for codecs. In your app, an example `EncodingConfig` with Protobuf as default codec might look like: - -```go -// MakeEncodingConfig creates an EncodingConfig -func MakeEncodingConfig() params.EncodingConfig { - amino := codec.NewLegacyAmino() - interfaceRegistry := types.NewInterfaceRegistry() - marshaler := codec.NewProtoCodec(interfaceRegistry) - txCfg := tx.NewTxConfig(marshaler, tx.DefaultSignModes) - - encodingConfig := params.EncodingConfig{ - InterfaceRegistry: interfaceRegistry, - Marshaler: marshaler, - TxConfig: txCfg, - Amino: amino, - } - std.RegisterLegacyAminoCodec(encodingConfig.Amino) - std.RegisterInterfaces(encodingConfig.InterfaceRegistry) - ModuleBasics.RegisterLegacyAminoCodec(encodingConfig.Amino) - ModuleBasics.RegisterInterfaces(encodingConfig.InterfaceRegistry) - - return encodingConfig -} -``` - -These codecs are used to populate the following fields on your app (again, we are using SimApp for demo purposes): - -+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/simapp/app.go#L146-L153 - -As explained in the [modules migration section](#updating-modules), some functions and structs in modules require an additional `codec.Codec` argument. You should pass `app.appCodec` in these cases, and this will be the default codec used throughout the app. - -### Registering Non-Module Protobuf Services - -We described in the [modules migration section](#updating-modules) `Query` and `Msg` services defined in each module. The SDK also exposes two more module-agnostic services: - -- the [Tx Service](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/proto/cosmos/tx/v1beta1/service.proto), to perform operations on transactions, -- the [Tendermint service](https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/proto/cosmos/base/tendermint/v1beta1/query.proto), to have a more idiomatic interface to the [Tendermint RPC](https://docs.tendermint.com/master/rpc/). - -These services are optional. To use them, or if you wish to add more module-agnostic Protobuf services into your app, you need to register them inside `app.go`: - -+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/simapp/app.go#L577-L585 - -### Registering `grpc-gateway` Routes - -The exising `RegisterAPIRoutes` method on the `app` only registers [Legacy API routes](../core/grpc_rest.md#legacy-rest-api-routes). If you are using `grpc-gateway` REST endpoints as described [above](#query-service), then these endpoints need to be wired up to a HTTP server: - -+++ https://github.com/cosmos/cosmos-sdk/blob/v0.40.0-rc6/simapp/app.go#L555-L575 - -## Next {hide} - -Learn how to perform a [chain upgrade](./chain-upgrade-guide-040.md) to 0.40. diff --git a/docs/migrations/chain-upgrade-guide-040.md b/docs/migrations/chain-upgrade-guide-040.md deleted file mode 100644 index 7a2848d3e7d3..000000000000 --- a/docs/migrations/chain-upgrade-guide-040.md +++ /dev/null @@ -1,166 +0,0 @@ - - -# Chain Upgrade Guide to v0.42 - -This document explains how to perform a chain upgrade from v0.39 to v0.42. {synopsis} - -::: tip -Please note that the three SDK versions v0.40, v0.41 and v0.42 are functionally equivalent, together called the "Stargate" series. The version bumps are consequences of post-release state-breaking bugfixes. -::: - -## Risks - -As a validator, performing the upgrade procedure on your consensus nodes carries a heightened risk of double-signing and -being slashed: if your validator node votes for a block, and, in the same block time, restarts the upgraded node, this may lead to double-voting on a block. - -The riskiest thing a validator can do is to discover that they made a mistake and repeat the upgrade procedure again during -the network startup. If you discover a mistake in the process, the best thing to do is wait for the network to start -before correcting it. If the network is halted and you have started with a different genesis file than the expected one, -seek advice from the validator community. - -## Recovery - -- Prior to exporting the state, the validators are encouraged to take a full data snapshot at exported height. Exported - height will be determined by a governance proposal. Data backup is usually done by copying daemon home directory, - e.g.: `~/.simd` - -**Note:** we use "simd" as our app throughout this doc, be sure to replace with the name of your own binary. - -It is critically important to back-up the validator state file, e.g.: `~/.simd/data/priv_validator_state.json` file -after stopping your daemon process. This file is updated every block as your validator participates in a consensus -rounds. It is a critical file needed to prevent double-signing, in case the upgrade fails, and the previous chain needs -to be restarted. - -In the event that the upgrade does not succeed, validators and operators must downgrade back to old version of the -software and restore to their latest snapshot before restarting their nodes. - -## Upgrade procedure - -1. The procedure is to export the state from the old binary, and import it with the new binary. First, verify your old binary version (which should use `github.com/cosmos/cosmos-sdk@0.39.*`) before exporting the state. - - ```shell - simd version --long - ``` - -1. Export the state from existing chain using the old binary. - - ```shell - simd export --for-zero-height --height > v039_exported_state.json - ``` - -1. Verify the SHA256 of the (sorted) exported genesis file: - - ```shell - $ jq -S -c -M '' v039_exported_state.json | shasum -a 256 - [SHASUM_PLACEHOLDER] v039_exported_state.json - ``` - -1. Cross check the hash with other peers (other validators) in the chat rooms. - -1. Install the latest binary (which uses `github.com/cosmos/cosmos-sdk@0.40.*`). - -1. Migrate the exported state to `github.com/cosmos/cosmos-sdk@0.40.*` compatible genesis state. - - ```shell - simd migrate v0.42 v039_exported_state.json --chain-id --genesis-time > new_v042_genesis.json - ``` - - **Note:** The migrate command takes an input genesis state and migrates it to a targeted version. New `genesis-time` is usually mentioned in the governance proposal, and should be passed as flag argument. If the flag is omitted, then the genesis time of the upgraded chain will be the same as the old one, which may cause confusion. - -1. All the necessary state changes are handled in the `simd migrate v0.42` migration command. However, Tendermint parameters are **not** handled in this command. You might need to update these parameters manually. - - In the recent versions of Tendermint, the following changes have been made: - - - `consensus_params.evidence.max_num` has been renamed to `consensus_params.evidence.max_bytes`. - - `consensus_params.evidence.max_age` has been removed, and replaced by `consensus_params.evidence.max_age_duration` and `consensus_params.evidence.max_age_num_blocks`. - - Make sure that your genesis JSON files contains the correct values specific to your chain. If the `simd migrate` errors with a message saying that the genesis file cannot be parsed, these are the fields to check first. - -1. Verify the SHA256 of the migrated genesis file with other validators to make sure there are no manual errors in the process. - - ```shell - $ jq -S -c -M '' new_v042_genesis.json | shasum -a 256 - [SHASUM_PLACEHOLDER] new_v042_genesis.json - ``` - -1. Make sure to update the genesis parameters in the new genesis if any. All these details will be generally present in - the governance proposal. - -1) If your chain is using IBC, make sure to add IBC initial genesis state to the genesis file. You can use the following command to add IBC initial genesis state to the genesis file. - - ```shell - cat new_v042_genesis.json | jq '.app_state |= . + {"ibc":{"client_genesis":{"clients":[],"clients_consensus":[],"create_localhost":false},"connection_genesis":{"connections":[],"client_connection_paths":[]},"channel_genesis":{"channels":[],"acknowledgements":[],"commitments":[],"receipts":[],"send_sequences":[],"recv_sequences":[],"ack_sequences":[]}},"transfer":{"port_id":"transfer","denom_traces":[],"params":{"send_enabled":false,"receive_enabled":false}},"capability":{"index":"1","owners":[]}}' > tmp_genesis.json && mv tmp_genesis.json new_v042_genesis.json - ``` - - **Note:** This would add IBC state with IBC's `send_enabled: false` and `receive_enabled: false`. Make sure to update them to `true` in the above command if are planning to enable IBC transactions with chain upgrade. Otherwise you can do it via a governance proposal. - -1) Reset the old state. - - **Note:** Be sure you have a complete backed up state of your node before proceeding with this step. - See Recovery for details on how to proceed. - - ```shell - simd unsafe-reset-all - ``` - -1) Move the new genesis.json to your daemon config directory. Ex - - ```shell - cp new_v042_genesis.json ~/.simd/config/genesis.json - ``` - -1) Update `~/.simd/config/app.toml` to include latest app configurations. [Here is the link](https://github.com/cosmos/cosmos-sdk/blob/v0.42.0-rc6/server/config/toml.go#L11-L164) to the default template for v0.42's `app.toml`. Make sure to - update your custom configurations as per your validator design, e.g. `gas_price`. - - Compared to v0.39, some notable updates to `app.toml` are: - - - API server is now configured to run in-process with daemon, previously it was a separate process, invoked by running rest-server - command i.e., `gaiacli rest-server`. Now it is in-process with daemon and can be enabled/disabled by API configuration: - - ```yaml - [api] - # Enable defines if the API server should be enabled. - enable = false - # Swagger defines if swagger documentation should automatically be registered. - swagger = false - ``` - - `swagger` setting refers to enabling/disabling swagger docs API, i.e, `/swagger/` API endpoint. - - - gRPC Configuration - - ```yaml - [grpc] - # Enable defines if the gRPC server should be enabled. - enable = true - # Address defines the gRPC server address to bind to. - address = "0.0.0.0:9090" - ``` - - - State Sync Configuration - - ```yaml - # State sync snapshots allow other nodes to rapidly join the network without replaying historical - # blocks, instead downloading and applying a snapshot of the application state at a given height. - [state-sync] - # snapshot-interval specifies the block interval at which local state sync snapshots are - # taken (0 to disable). Must be a multiple of pruning-keep-every. - snapshot-interval = 0 - # snapshot-keep-recent specifies the number of recent snapshots to keep and serve (0 to keep all). - snapshot-keep-recent = 2 - ``` - -1) Kill if any external `rest-server` process is running. - -1) All set now! You can (re)start your daemon to validate on the upgraded network. Make sure to check your binary version - before starting the daemon: - - ``` - simd version --long - ``` - -## Next {hide} - -Once your chain is upgraded, make sure to [update your clients' REST endpoints](./rest.md). diff --git a/docs/migrations/chain-upgrade-guide-043.md b/docs/migrations/chain-upgrade-guide-043.md new file mode 100644 index 000000000000..129d22314c59 --- /dev/null +++ b/docs/migrations/chain-upgrade-guide-043.md @@ -0,0 +1,237 @@ + + +# Chain Upgrade Guide to v0.43 + +This document provides guidelines for a chain upgrade from v0.42 to v0.43 and an example of the upgrade process using `simapp`. {synopsis} + +::: tip +You must upgrade to Stargate v0.42 before upgrading to v0.43. If you have not done so, please see [Chain Upgrade Guide to v0.42](/v0.42/migrations/chain-upgrade-guide-040.html). +::: + +## Prerequisite Readings + +- [Upgrading Modules](../building-modules/upgrade.html) {prereq} +- [In-Place Store Migrations](../core/upgrade.html) {prereq} +- [Cosmovisor](../run-node/cosmovisor.html) {prereq} + +Cosmos SDK v0.43 introduces a new way of handling chain upgrades that no longer requires exporting state to JSON, making the necesssary changes, and then creating a new chain with the modified JSON as the new genesis file. + +Instead of starting a new chain, the upgrade binary will read the existing database and perform in-place store migrations. This new way of handling chain upgrades can be used alongside [Cosmovisor](../run-node/cosmovisor.html) to make the upgrade process seamless. + +## In-Place Store Migrations + +We recommend using [In-Place Store Migrations](../core/upgrade.html) to upgrade your chain from v0.42 to v0.43. The first step is to make sure all your modules follow the [Module Upgrade Guide](../building-modules/upgrade.html). The second step is to add an [upgrade handler](../core/upgrade.html#running-migrations) to `app.go`. + +In this document, we'll provide an example of what the upgrade handler looks like for a chain upgrading module versions for the first time. It's critical to note that the initial version of each module must be set to `1` rather than `0` or else the upgrade handler will re-initialize each module. + +In addition to migrating existing modules, the upgrade handler also performs store upgrades for new modules. In the example below, we'll be adding store migrations for two new modules made available in v0.43: `x/authz` and `x/feegrant`. + +## Using Cosmovisor + +We recommend validators use [Cosmovisor](../run-node/cosmovisor.html), which is a process manager for running application binaries. For security reasons, we recommend validators build their own upgrade binaries rather than enabling the auto-download option. Validators may still choose to use the auto-download option if the necessary security guarantees are in place (i.e. the URL provided in the upgrade proposal for the downloadable upgrade binary includes a proper checksum). + +Validators can use the auto-restart option to prevent unecessary downtime during the upgrade process. The auto-restart option will automatically restart the chain with the upgrade binary once the chain has halted at the proposed upgrade height. With the auto-restart option, validators can prepare the upgrade binary in advance and then relax at the time of the upgrade. + +## Migrating app.toml + +With the update to `v0.43`, new server configuration options have been added to `app.toml`. The updates include new configuration sections for Rosetta and gRPC Web as well as a new configuration option for State Sync. Check out the default [`app.toml`](https://github.com/cosmos/cosmos-sdk/blob/release/v0.43.x/server/config/toml.go) file in the latest version of `v0.43` for more information. + +## Example: Simapp Upgrade + +The following example will walk through the upgrade process using `simapp` as our blockchain application. We will be upgrading `simapp` from v0.42 to v0.43. We will be building the upgrade binary ourselves and enabling the auto-restart option. + +*Note: In this example, we will be starting a new chain from `v0.42`. The binary for this version will be the genesis binary. For validators using Cosmovisor for the first time, the binary for the current version of the chain should be used as the genesis binary (i.e. the starting binary). For more information, see [Cosmovisor](../run-node/cosmovisor.html).* + +### Initial Setup + +From within the `cosmos-sdk` repository, check out the latest `v0.42.x` release: + +``` +git checkout release/v0.42.x +``` + +Build the `simd` binary for the latest `v0.42.x` release (the genesis binary): + +``` +make build +``` + +Reset `~/.simapp` (never do this in a production environment): + +``` +./build/simd unsafe-reset-all +``` + +Configure the `simd` binary for testing: + +``` +./build/simd config chain-id test +./build/simd config keyring-backend test +./build/simd config broadcast-mode block +``` + +Initialize the node and overwrite any previous genesis file (never do this in a production environment): + + + +``` +./build/simd init test --chain-id test --overwrite +``` + +Set the minimum gas price to `0stake` in `~/.simapp/config/app.toml`: + +``` +minimum-gas-prices = "0stake" +``` + +For the purpose of this demonstration, change `voting_period` in `genesis.json` to a reduced time of 20 seconds (`20s`): + +``` +cat <<< $(jq '.app_state.gov.voting_params.voting_period = "20s"' $HOME/.simapp/config/genesis.json) > $HOME/.simapp/config/genesis.json +``` + +Create a new key for the validator, then add a genesis account and transaction: + + + + +``` +./build/simd keys add validator +./build/simd add-genesis-account validator 5000000000stake --keyring-backend test +./build/simd gentx validator 1000000stake --chain-id test +./build/simd collect-gentxs +``` + +Now that our node is initialized and we are ready to start a new `simapp` chain, let's set up `cosmovisor` and the genesis binary. + +### Cosmovisor Setup + +First, install or update `cosmovisor`: + +``` +go get github.com/cosmos/cosmos-sdk/cosmovisor/cmd/cosmovisor +``` + +Set the required environment variables: + +``` +export DAEMON_NAME=simd +export DAEMON_HOME=$HOME/.simapp +``` + +Set the optional environment variable to trigger an automatic restart: + +``` +export DAEMON_RESTART_AFTER_UPGRADE=true +``` + +Create the folder for the genesis binary and copy the `v0.42.x` binary: + +``` +mkdir -p $DAEMON_HOME/cosmovisor/genesis/bin +cp ./build/simd $DAEMON_HOME/cosmovisor/genesis/bin +``` + +Now that `cosmovisor` is installed and the genesis binary has been added, let's add the upgrade handler to `simapp/app.go` and prepare the upgrade binary. + +### Chain Upgrade + + + +Check out `release/v0.43.x`: + +``` +git checkout release/v0.43.x +``` + +Add the following to `simapp/app.go` inside `NewSimApp` and after `app.UpgradeKeeper`: + +```go + app.registerUpgradeHandlers() +``` + +Add the following to `simapp/app.go` after `NewSimApp` (to learn more about the upgrade handler, see the [In-Place Store Migrations](../core/upgrade.html)): + +```go +func (app *SimApp) registerUpgradeHandlers() { + app.UpgradeKeeper.SetUpgradeHandler("v0.43", func(ctx sdk.Context, plan upgradetypes.Plan, _ module.VersionMap) (module.VersionMap, error) { + // 1st-time running in-store migrations, using 1 as fromVersion to + // avoid running InitGenesis. + fromVM := map[string]uint64{ + "auth": 1, + "bank": 1, + "capability": 1, + "crisis": 1, + "distribution": 1, + "evidence": 1, + "gov": 1, + "mint": 1, + "params": 1, + "slashing": 1, + "staking": 1, + "upgrade": 1, + "vesting": 1, + "ibc": 1, + "genutil": 1, + "transfer": 1, + } + + return app.mm.RunMigrations(ctx, app.configurator, fromVM) + }) + + upgradeInfo, err := app.UpgradeKeeper.ReadUpgradeInfoFromDisk() + if err != nil { + panic(err) + } + + if upgradeInfo.Name == "v0.43" && !app.UpgradeKeeper.IsSkipHeight(upgradeInfo.Height) { + storeUpgrades := storetypes.StoreUpgrades{ + Added: []string{"authz", "feegrant"}, + } + + // configure store loader that checks if version == upgradeHeight and applies store upgrades + app.SetStoreLoader(upgradetypes.UpgradeStoreLoader(upgradeInfo.Height, &storeUpgrades)) + } +} +``` + +Add `storetypes` to imports: + +```go + storetypes "github.com/cosmos/cosmos-sdk/store/types" +``` + +Build the `simd` binary for `v0.43.x` (the upgrade binary): + +``` +make build +``` + +Create the folder for the upgrade binary and copy the `v0.43.x` binary: + +``` +mkdir -p $DAEMON_HOME/cosmovisor/upgrades/v0.43/bin +cp ./build/simd $DAEMON_HOME/cosmovisor/upgrades/v0.43/bin +``` + +Now that we have added the upgrade handler and prepared the upgrade binary, we are ready to start `cosmovisor` and simulate the upgrade proposal process. + +### Upgrade Proposal + +Start the node using `cosmovisor`: + +``` +cosmovisor start +``` + +Open a new terminal window and submit an upgrade proposal along with a deposit and a vote (these commands must be run within 20 seconds of each other): + +``` +./build/simd tx gov submit-proposal software-upgrade v0.43 --title upgrade --description upgrade --upgrade-height 20 --from validator --yes +./build/simd tx gov deposit 1 10000000stake --from validator --yes +./build/simd tx gov vote 1 yes --from validator --yes +``` + +Confirm the chain automatically upgrades at height 20. diff --git a/docs/migrations/keyring.md b/docs/migrations/keyring.md deleted file mode 100644 index 4fe3a3ebc03f..000000000000 --- a/docs/migrations/keyring.md +++ /dev/null @@ -1,33 +0,0 @@ - - -# Keyring Migrate Quick Start - -`keyring` is the Cosmos SDK mechanism to manage the public/private keypair. Cosmos SDK v0.42 (Stargate) introduced breaking changes in the keyring. - -To upgrade your chain from v0.39 (Launchpad) and earlier to Stargate, you must migrate your keys inside the keyring to the latest version. For details on configuring and using the keyring, see [Setting up the keyring](../run-node/keyring.md). - -This guide describes how to migrate your keyrings. - -The following command migrates your keyrings: - -```bash -Usage -simd keys migrate -``` - -The migration process moves key information from the legacy db-based Keybase to the [keyring](https://github.com/99designs/keyring)-based Keyring. The legacy Keybase persists keys in a LevelDB database in a 'keys' sub-directory of the client application home directory (`old_home_dir`). For example, `$HOME/.gaiacli/keys/` for [Gaia](https://github.com/cosmos/gaia). - -You can migrate or skip the migration for each key entry found in the specified `old_home_dir` directory. Each key migration requires a valid passphrase. If an invalid passphrase is entered, the command exits. Run the command again to restart the keyring migration. - -The `migrate` command takes the following flags: - -- `--dry-run` boolean - - - true - run the migration but do not persist changes to the new Keybase. - - false - run the migration and persist keys to the new Keybase. - -Recommended: Use `--dry-run true` to test the migration without persisting changes before you migrate and persist keys. - -- `--keyring-backend` string flag. It allows you to select a backend. For more detailed information about the available backends, you can read [the keyring guide](../run-node/keyring.md). diff --git a/docs/migrations/rest.md b/docs/migrations/rest.md index 93f890a7718b..9978974a2730 100644 --- a/docs/migrations/rest.md +++ b/docs/migrations/rest.md @@ -1,14 +1,14 @@ # REST Endpoints Migration -Migrate your REST endpoints to the Stargate ones. {synopsis} +Migrate your REST endpoints. Legacy REST endpoints have been deprecated since v0.40 and they will be removed in v0.44. {synopsis} ## Deprecation of Legacy REST Endpoints -The Cosmos SDK versions v0.39 and earlier provided REST endpoints to query the state and broadcast transactions. These endpoints are kept in Cosmos SDK v0.40 (Stargate), but they are marked as deprecated, and will be removed in v0.41. We therefore call these endpoints legacy REST endpoints. +The Cosmos SDK versions v0.39 and earlier provided REST endpoints to query the state and broadcast transactions. These endpoints were kept in Cosmos SDK v0.40 and they are still available in v0.43, but they are marked as deprecated and will be removed in v0.44. We therefore call these endpoints legacy REST endpoints. Some important information concerning all legacy REST endpoints: @@ -32,9 +32,9 @@ Some important information concerning all legacy REST endpoints: ## Migrating to New REST Endpoints -Thanks to the Protocol Buffers migration in v0.40, we are able to take advantage of a vast number of gRPC tools and solutions. For most of the legacy REST endpoints, Cosmos SDK v0.40 provides new REST endpoints generated from [gRPC `Query` services](../building-modules/query-services.md) using [grpc-gateway](https://grpc-ecosystem.github.io/grpc-gateway/). We usually call them _gRPC-gateway REST endpoints_. +Thanks to the Protocol Buffers migration in v0.40, we are able to take advantage of a vast number of gRPC tools and solutions. Most of the legacy REST endpoints have been replaced by REST endpoints generated from [gRPC `Query` services](../building-modules/query-services.md) using [grpc-gateway](https://grpc-ecosystem.github.io/grpc-gateway/). We usually call them _gRPC-gateway REST endpoints_. -Some modules expose legacy `POST` endpoints to generate unsigned transactions for their `Msg`s. These `POST` endpoints have been removed. We recommend to use [service `Msg`s](../building-modules/msg-services.md) directly, and use Protobuf to do client-side transaction generation. A guide can be found [here](../run-node/txs.md). +Previously, some modules exposed legacy `POST` endpoints to generate unsigned transactions for their `Msg`s. These `POST` endpoints were removed in v0.40. We recommend using [Protobuf `Msg` service](../building-modules/msg-services.md) directly to do client-side transaction generation. A guide can be found [here](../run-node/txs.md). | Legacy REST Endpoint | Description | New gRPC-gateway REST Endpoint | | ------------------------------------------------------------------------------- | ------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |