Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: ADR-040: Refactor App to use v2.MultiStore #10174

Closed
wants to merge 93 commits into from
Closed
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
a549522
[WIP] refactor multistore => root store
roysc Aug 31, 2021
ec5d41e
baseapp - disable snapshot tests
roysc Oct 27, 2021
9db21a5
store updates, fixes
roysc Jan 20, 2022
ceabd08
updates, rf
roysc Jan 18, 2022
ab531ff
changelog + docs
roysc Jan 20, 2022
47330c5
Merge remote-tracking branch 'upstream/master' into refactor-multistore
roysc Jan 20, 2022
929f5ad
fixes
roysc Jan 20, 2022
3876113
Merge branch 'master' into roysc/adr-040-refactor-multistore
roysc Jan 20, 2022
25d0a9f
changelog fix
roysc Jan 21, 2022
355f181
cleanup
roysc Jan 21, 2022
8cbf4b2
nit, StoreConfig =>StoreParams
roysc Jan 21, 2022
4e431eb
store doc
roysc Jan 21, 2022
861b24c
store rearrange
roysc Jan 21, 2022
f45ce0f
godoc nits
roysc Jan 27, 2022
79bea1d
rm StoreConstructor
roysc Jan 28, 2022
e912111
compatibility wrapper for original MultiStore interface
roysc Jan 27, 2022
704570b
rename BasicMultiStore -> MultiStore
roysc Feb 7, 2022
b36c8f5
patch multistore
roysc Feb 7, 2022
d035fb8
patch compat store
roysc Feb 7, 2022
bfa7b88
multi store- use StoreKey instances, not strings
roysc Feb 14, 2022
ccb908d
wrap v1 MultiStore in v2 interface; use in baseapp
roysc Feb 7, 2022
532a6ad
viewstore.CacheStore() replaces CacheMultiStoreWithVersion
roysc Feb 15, 2022
5f6fe12
test cleanup
roysc Feb 15, 2022
b200867
trace context fence
roysc Feb 15, 2022
9cdd990
get all versions
roysc Feb 15, 2022
2c59d34
Merge branch 'master' of github.com:cosmos/cosmos-sdk into refactor-m…
roysc Feb 15, 2022
2627ce3
Merge remote-tracking branch 'upstream/master' into refactor-multistore
roysc Feb 15, 2022
555be73
patches
roysc Feb 18, 2022
b631a9a
Merge branch 'master' of github.com:cosmos/cosmos-sdk into refactor-m…
roysc Feb 18, 2022
83cd109
patches
roysc Feb 18, 2022
71a9bb9
fix v1asv2 commit
roysc Feb 18, 2022
78b0a12
PR revisions
roysc Feb 23, 2022
ddf8d0f
fix simd export
roysc Feb 23, 2022
3ddf675
open db at data/application/
roysc Feb 23, 2022
def4107
db cleanup
roysc Feb 23, 2022
880a4ee
badgerdb fix
roysc Feb 23, 2022
8f58686
Merge branch 'master' of github.com:cosmos/cosmos-sdk into refactor-m…
roysc Feb 25, 2022
fcb9ff9
Merge remote-tracking branch 'upstream/master' into refactor-multistore
roysc Feb 25, 2022
332c3e9
change rocksdb build tag to "rocksdb"
roysc Mar 2, 2022
d7a51f5
badgerdb - handle ErrTxnTooBig
roysc Mar 2, 2022
5102f92
remove multistore reverse index
roysc Mar 2, 2022
81642a5
rename tmdb adapter
roysc Mar 4, 2022
b33b162
GetAllVersions patch
roysc Mar 4, 2022
df2e47b
db/adapter - return discard error
roysc Mar 5, 2022
1dd7de8
multistore - minor additions
roysc Mar 10, 2022
4c34467
spelling
roysc Mar 16, 2022
44fafb2
Merge remote-tracking branch 'upstream/master' into refactor-multistore
roysc Mar 16, 2022
86929d3
badgerdb: fix versions file write
roysc Mar 25, 2022
4cdd5ee
version manager - DeleteAbove() & implement w/ slice
roysc Mar 24, 2022
e36e3ad
DBConnection.RevertTo
roysc Mar 24, 2022
f0d31ba
unit test badgerdb.versionmanager
roysc Mar 25, 2022
ae5c007
Merge remote-tracking branch 'upstream/master' into refactor-multistore
roysc Mar 25, 2022
620728f
refactor multistore: fix state sync
roysc Mar 25, 2022
9e2152e
refactor multistore: fix store migration
roysc Mar 27, 2022
68da759
fix rename store/{v2 => v2alpha1}
roysc Mar 27, 2022
bae2e7a
dbmapstore.Delete should raise smt.InvalidKeyError
roysc Mar 30, 2022
138bd43
Merge remote-tracking branch 'upstream/master' into refactor-multistore
roysc Apr 7, 2022
1295d81
dispatching db constructor
roysc Apr 7, 2022
fcf64ed
refactor server rollback cmd
roysc Apr 7, 2022
4dd37ca
go mod tidy
roysc Apr 7, 2022
7eb1f9e
fix app-db-backend config
roysc Apr 8, 2022
0499da2
comment typo
roysc Apr 12, 2022
924d2ec
reorg multistore tests
roysc Apr 12, 2022
7f747ab
check multistore key types
roysc Apr 12, 2022
7de3adf
fix v1asv2 methods
roysc Apr 12, 2022
47ab63f
tests and fixes for v1asv2 adapter
roysc Apr 12, 2022
1568198
force mem & transient stores to use bespoke key type
roysc Apr 12, 2022
bb15cd3
godoc copyedit
roysc Apr 19, 2022
e103a63
smt store test nits
roysc Apr 19, 2022
1a37cc2
[dev] update SMT library
roysc Apr 18, 2022
7b1e2f3
Merge remote-tracking branch 'upstream/main' into refactor-multistore
roysc May 30, 2022
db91469
Patch merge
roysc Jun 2, 2022
5c1b566
Patch simapp upgrade handler
roysc Jun 2, 2022
5f83976
Revert "[dev] update SMT library"
roysc May 31, 2022
2ad0ee0
Merge remote-tracking branch 'upstream/main' into refactor-multistore
roysc Jun 14, 2022
0b9aadc
put pruning manager into v2/multi store
roysc Jun 23, 2022
538bacb
fixup renames
roysc Jul 16, 2022
d7ff909
fix db creators - refactor packages/build directives
roysc Jul 4, 2022
04c832f
clean up v2 store re-exports
roysc Jun 27, 2022
1232d5a
v2 store additions
roysc Jun 28, 2022
f007704
add multi.Store.HasKVStore()
roysc Jun 24, 2022
bc67cfe
bank keeper: validate account store address
roysc Jun 29, 2022
e409ca3
fix typos, docs, rm redundant
roysc Jun 29, 2022
4823d1a
feat: list snapshots query (#246) (#252)
roysc Jul 20, 2022
69fd2f7
Restore baseapp pruning & snapshot tests
roysc Jun 22, 2022
d3f6289
pruning manager - decouple db, rm logger, fix test
roysc Jun 23, 2022
0f24e1e
Merge remote-tracking branch 'upstream/main' into refactor-multistore
roysc Jul 20, 2022
23d1a92
Merge branch 'main' into refactor-multistore
roysc Jul 21, 2022
db3c28a
fix merge
roysc Jul 20, 2022
eaa83be
go.mod/sum
roysc Jul 20, 2022
c1e800c
PR clean up
roysc Jul 21, 2022
1a9e194
v2 store: allow shared substore name prefixes
roysc Jul 21, 2022
24dd6c0
Merge remote-tracking branch 'upstream/main' into refactor-multistore
roysc Jul 22, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ Ref: https://keepachangelog.com/en/1.0.0/
* [\#10507](https://github.com/cosmos/cosmos-sdk/pull/10507) Add middleware for tx priority.
* [\#10311](https://github.com/cosmos/cosmos-sdk/pull/10311) Adds cli to use tips transactions. It adds an `--aux` flag to all CLI tx commands to generate the aux signer data (with optional tip), and a new `tx aux-to-fee` subcommand to let the fee payer gather aux signer data and broadcast the tx
* [\#10430](https://github.com/cosmos/cosmos-sdk/pull/10430) ADR-040: Add store/v2 `MultiStore` implementation
* [\#10174](https://github.com/cosmos/cosmos-sdk/pull/10174) ADR-040: Refactor App to use `v2.MultiStore`

### API Breaking Changes

Expand Down
24 changes: 13 additions & 11 deletions baseapp/abci.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import (

"github.com/cosmos/cosmos-sdk/codec"
snapshottypes "github.com/cosmos/cosmos-sdk/snapshots/types"
"github.com/cosmos/cosmos-sdk/store/v2/multi"
sdk "github.com/cosmos/cosmos-sdk/types"
sdkerrors "github.com/cosmos/cosmos-sdk/types/errors"
"github.com/cosmos/cosmos-sdk/types/tx"
Expand All @@ -34,7 +35,7 @@ func (app *BaseApp) InitChain(req abci.RequestInitChain) (res abci.ResponseInitC
if req.InitialHeight > 1 {
app.initialHeight = req.InitialHeight
initHeader = tmproto.Header{ChainID: req.ChainId, Height: req.InitialHeight, Time: req.Time}
err := app.cms.SetInitialVersion(req.InitialHeight)
err := app.store.SetInitialVersion(uint64(req.InitialHeight))
if err != nil {
panic(err)
}
Expand Down Expand Up @@ -104,7 +105,7 @@ func (app *BaseApp) InitChain(req abci.RequestInitChain) (res abci.ResponseInitC

// Info implements the ABCI interface.
func (app *BaseApp) Info(req abci.RequestInfo) abci.ResponseInfo {
lastCommitID := app.cms.LastCommitID()
lastCommitID := app.store.LastCommitID()

return abci.ResponseInfo{
Data: app.name,
Expand Down Expand Up @@ -136,8 +137,8 @@ func (app *BaseApp) FilterPeerByID(info string) abci.ResponseQuery {
// BeginBlock implements the ABCI application interface.
func (app *BaseApp) BeginBlock(req abci.RequestBeginBlock) (res abci.ResponseBeginBlock) {

if app.cms.TracingEnabled() {
app.cms.SetTracingContext(sdk.TraceContext(
if app.store.TracingEnabled() {
app.store.SetTracingContext(sdk.TraceContext(
map[string]interface{}{"blockHeight": req.Header.Height},
))
}
Expand Down Expand Up @@ -203,7 +204,7 @@ func (app *BaseApp) BeginBlock(req abci.RequestBeginBlock) (res abci.ResponseBeg
func (app *BaseApp) EndBlock(req abci.RequestEndBlock) (res abci.ResponseEndBlock) {

if app.deliverState.ms.TracingEnabled() {
app.deliverState.ms = app.deliverState.ms.SetTracingContext(nil).(sdk.CacheMultiStore)
app.deliverState.ms.SetTracingContext(nil)
}

if app.endBlocker != nil {
Expand Down Expand Up @@ -306,9 +307,9 @@ func (app *BaseApp) Commit() (res abci.ResponseCommit) {

// Write the DeliverTx state into branched storage and commit the MultiStore.
// The write to the DeliverTx state writes all state transitions to the root
// MultiStore (app.cms) so when Commit() is called is persists those values.
// MultiStore (app.store) so when Commit() is called it persists those values.
app.deliverState.ms.Write()
commitID := app.cms.Commit()
commitID := app.store.Commit()
app.logger.Info("commit synced", "commit", fmt.Sprintf("%X", commitID))

// Reset the Check state to the latest committed.
Expand Down Expand Up @@ -637,14 +638,15 @@ func (app *BaseApp) createQueryContext(height int64, prove bool) (sdk.Context, e
)
}

cacheMS, err := app.cms.CacheMultiStoreWithVersion(height)
version, err := app.store.GetVersion(height)
roysc marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
return sdk.Context{},
sdkerrors.Wrapf(
sdkerrors.ErrInvalidRequest,
"failed to load state at height %d; %s (latest height: %d)", height, err, app.LastBlockHeight(),
)
}
cacheMS := multi.BasicAsCacheStore(version)

// branch the commit-multistore for safety
ctx := sdk.NewContext(
Expand Down Expand Up @@ -711,7 +713,7 @@ func (app *BaseApp) GetBlockRetentionHeight(commitHeight int64) int64 {

// Define the state pruning offset, i.e. the block offset at which the
// underlying logical database is persisted to disk.
statePruningOffset := int64(app.cms.GetPruning().KeepEvery)
statePruningOffset := int64(app.store.GetPruning().KeepEvery)
if statePruningOffset > 0 {
if commitHeight > statePruningOffset {
v := commitHeight - (commitHeight % statePruningOffset)
Expand Down Expand Up @@ -789,12 +791,12 @@ func handleQueryApp(app *BaseApp, path []string, req abci.RequestQuery) abci.Res

func handleQueryStore(app *BaseApp, path []string, req abci.RequestQuery) abci.ResponseQuery {
// "/store" prefix for store queries
queryable, ok := app.cms.(sdk.Queryable)
queryable, ok := app.store.(sdk.Queryable)
if !ok {
return sdkerrors.QueryResult(sdkerrors.Wrap(sdkerrors.ErrUnknownRequest, "multistore doesn't support queries"), app.trace)
}

req.Path = "/" + strings.Join(path[1:], "/")
req.Path = "/" + strings.Join(path[1:], "/") // = /store/main/key -> /main/key

if req.Height <= 1 && req.Prove {
return sdkerrors.QueryResult(
Expand Down
25 changes: 13 additions & 12 deletions baseapp/abci_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,14 @@ import (
"github.com/stretchr/testify/require"
abci "github.com/tendermint/tendermint/abci/types"
tmprototypes "github.com/tendermint/tendermint/proto/tendermint/types"
dbm "github.com/tendermint/tm-db"

"github.com/cosmos/cosmos-sdk/baseapp"
"github.com/cosmos/cosmos-sdk/db/memdb"
sdk "github.com/cosmos/cosmos-sdk/types"
)

func TestGetBlockRentionHeight(t *testing.T) {
logger := defaultLogger()
db := dbm.NewMemDB()
name := t.Name()

testCases := map[string]struct {
Expand All @@ -25,20 +24,20 @@ func TestGetBlockRentionHeight(t *testing.T) {
expected int64
}{
"defaults": {
bapp: baseapp.NewBaseApp(name, logger, db),
bapp: baseapp.NewBaseApp(name, logger, memdb.NewDB()),
maxAgeBlocks: 0,
commitHeight: 499000,
expected: 0,
},
"pruning unbonding time only": {
bapp: baseapp.NewBaseApp(name, logger, db, baseapp.SetMinRetainBlocks(1)),
bapp: baseapp.NewBaseApp(name, logger, memdb.NewDB(), baseapp.SetMinRetainBlocks(1)),
maxAgeBlocks: 362880,
commitHeight: 499000,
expected: 136120,
},
"pruning iavl snapshot only": {
bapp: baseapp.NewBaseApp(
name, logger, db,
name, logger, memdb.NewDB(),
baseapp.SetPruning(sdk.PruningOptions{KeepEvery: 10000}),
baseapp.SetMinRetainBlocks(1),
),
Expand All @@ -48,7 +47,7 @@ func TestGetBlockRentionHeight(t *testing.T) {
},
"pruning state sync snapshot only": {
bapp: baseapp.NewBaseApp(
name, logger, db,
name, logger, memdb.NewDB(),
baseapp.SetSnapshotInterval(50000),
baseapp.SetSnapshotKeepRecent(3),
baseapp.SetMinRetainBlocks(1),
Expand All @@ -59,7 +58,7 @@ func TestGetBlockRentionHeight(t *testing.T) {
},
"pruning min retention only": {
bapp: baseapp.NewBaseApp(
name, logger, db,
name, logger, memdb.NewDB(),
baseapp.SetMinRetainBlocks(400000),
),
maxAgeBlocks: 0,
Expand All @@ -68,7 +67,7 @@ func TestGetBlockRentionHeight(t *testing.T) {
},
"pruning all conditions": {
bapp: baseapp.NewBaseApp(
name, logger, db,
name, logger, memdb.NewDB(),
baseapp.SetPruning(sdk.PruningOptions{KeepEvery: 10000}),
baseapp.SetMinRetainBlocks(400000),
baseapp.SetSnapshotInterval(50000), baseapp.SetSnapshotKeepRecent(3),
Expand All @@ -79,7 +78,7 @@ func TestGetBlockRentionHeight(t *testing.T) {
},
"no pruning due to no persisted state": {
bapp: baseapp.NewBaseApp(
name, logger, db,
name, logger, memdb.NewDB(),
baseapp.SetPruning(sdk.PruningOptions{KeepEvery: 10000}),
baseapp.SetMinRetainBlocks(400000),
baseapp.SetSnapshotInterval(50000), baseapp.SetSnapshotKeepRecent(3),
Expand All @@ -90,7 +89,7 @@ func TestGetBlockRentionHeight(t *testing.T) {
},
"disable pruning": {
bapp: baseapp.NewBaseApp(
name, logger, db,
name, logger, memdb.NewDB(),
baseapp.SetPruning(sdk.PruningOptions{KeepEvery: 10000}),
baseapp.SetMinRetainBlocks(0),
baseapp.SetSnapshotInterval(50000), baseapp.SetSnapshotKeepRecent(3),
Expand All @@ -104,7 +103,8 @@ func TestGetBlockRentionHeight(t *testing.T) {
for name, tc := range testCases {
tc := tc

tc.bapp.SetParamStore(&paramStore{db: dbm.NewMemDB()})
tc.bapp.SetParamStore(newParamStore(memdb.NewDB()))
require.NoError(t, tc.bapp.Init())
tc.bapp.InitChain(abci.RequestInitChain{
ConsensusParams: &tmprototypes.ConsensusParams{
Evidence: &tmprototypes.EvidenceParams{
Expand All @@ -116,6 +116,7 @@ func TestGetBlockRentionHeight(t *testing.T) {
t.Run(name, func(t *testing.T) {
require.Equal(t, tc.expected, tc.bapp.GetBlockRetentionHeight(tc.commitHeight))
})
require.NoError(t, tc.bapp.CloseStore())
}
}

Expand All @@ -125,7 +126,7 @@ func TestBaseAppCreateQueryContextRejectsNegativeHeights(t *testing.T) {
t.Parallel()

logger := defaultLogger()
db := dbm.NewMemDB()
db := memdb.NewDB()
name := t.Name()
app := baseapp.NewBaseApp(name, logger, db)

Expand Down
Loading