-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: ADR-040: add state sync for v2 store #10794
feat: ADR-040: add state sync for v2 store #10794
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, except the whitespace changes. (Note - reviewed internally at vulcanize#18)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few small changes requested
…cosmos-sdk into sai/10194_state_sync_v2_store
@i-norden Can you please review again, I addressed the pr comments |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM Sai!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's find a way to reduce copy-paste
store/v2/multi/snapshot.go
Outdated
return sdkerrors.Wrapf(err, "error while getting the snapshot versions at height %v", height) | ||
} | ||
if !versions.Exists(height) { | ||
return sdkerrors.Wrapf(sdkerrors.ErrNotFound, "cannot find snapshot at height %v", height) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this will be covered by an error from getView
, we actually don't need to access statedb.Versions()
at all in these tests
// get the saved snapshot at height | ||
vs, err := rs.getView(int64(height)) | ||
if err != nil { | ||
return sdkerrors.Wrap(err, fmt.Sprintf("error while get the version at height %d", height)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's add a case to cover this error
store/v2/multi/snapshot.go
Outdated
return strings.Compare(sKeys[i], sKeys[j]) == -1 | ||
}) | ||
|
||
var storeByteKeys [][]byte |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could just use this and remove skeys
store/v2/multi/snapshot.go
Outdated
if err != nil { | ||
return snapshottypes.SnapshotItem{}, sdkerrors.Wrapf(err, "error while getting the snapshot versions at height %v", height) | ||
} | ||
if versions.Count() != 0 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead, better to check LastCommitID().Version == 0
, then we can remove versions
.
And lets add a testcase for this
store/v2/multi/snapshot.go
Outdated
|
||
switch item := snapshotItem.Item.(type) { | ||
case *snapshottypes.SnapshotItem_Schema: | ||
if len(rs.schema) != 0 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We will actually need to change this behavior. StoreKeys are registered by the app at init, and exist as unique objects referenced only from modules that have access to them. (I recently corrected the v2 store to follow this pattern.)
So, the store needs to have a schema containing all the stores received by the snapshot before the snapshot is received - this is the current behavior for the current MultiStore.
So here, instead of overwriting the store's schema, we'll just check that it matches the one received.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
okay
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. It would be good to add some more failure test cases, like out-of-order messages. But no need to block on that.
store/v2/multi/snapshot.go
Outdated
return snapshottypes.SnapshotItem{}, sdkerrors.Wrap(err, "error at set the store schema key values") | ||
} | ||
if !rs.schema.equal(receivedStoreSchema) { | ||
return snapshottypes.SnapshotItem{}, sdkerrors.Wrap(sdkerrors.ErrLogic, "received store schema is not matched with app schema empty") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"received schema does not match app schema"
|
||
// ValidRestoreHeight will check height is valid for snapshot restore or not | ||
func ValidRestoreHeight(format uint32, height uint64) error { | ||
if format != snapshottypes.CurrentFormat { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should increase the format number?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, I forgot to mention this. I wonder if we should also change it for the snapshot extensions support?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
but CurrentFormat
is static , may be we need to change the CurrentFormat
is configurable
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
snapshot extensions feature is backward compatible if the app doesn't actually register any extensions, so I think it's good there.
but CurrentFormat is static , may be we need to change the CurrentFormat is configurable
Just bump the value of the constant? it's for the receiver to fail fast when receiving an unsupported format.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gsk967 , could you go thorough the comments and resolve them?
done
store/v2/multi/snapshot.go
Outdated
subStore.Set(item.KV.Key, item.KV.Value) | ||
|
||
default: | ||
return snapshottypes.SnapshotItem{}, sdkerrors.Wrapf(sdkerrors.ErrLogic, "unknown snapshot item %T", item) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When see an unknown item, we should treat it as a success, and return that unknown item to the snapshot manager, because that could be the beginning of the next extension snapshotter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I fixed a similar issue in rootmulti store's Restore
implementation: #11286
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks i fixed that one now , can review again changes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When restoration sees an unknown item, it should treat it as the ending of the current snapshot, and return that item to the snapshot manager, because it could be the beginning of the next extension snapshot.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@gsk967 , could you go thorough the comments and resolve them? |
@robert-zaremba done |
Ian/10194 state sync v2 store
## Description Closes: cosmos#10705 State Sync for V2 Store (ADR-40) --- ### Author Checklist *All items are required. Please add a note to the item if the item is not applicable and please add links to any relevant follow up issues.* I have... - [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] added `!` to the type prefix if API or client breaking change - [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting)) - [ ] provided a link to the relevant issue or specification - [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules) - [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing) - [ ] added a changelog entry to `CHANGELOG.md` - [ ] included comments for [documenting Go code](https://blog.golang.org/godoc) - [ ] updated the relevant documentation or specification - [ ] reviewed "Files changed" and left comments if necessary - [ ] confirmed all CI checks have passed ### Reviewers Checklist *All items are required. Please add a note if the item is not applicable and please add your handle next to the items reviewed if you only reviewed selected items.* I have... - [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] confirmed `!` in the type prefix if API or client breaking change - [ ] confirmed all author checklist items have been addressed - [ ] reviewed state machine logic - [ ] reviewed API design and naming - [ ] reviewed documentation is accurate - [ ] reviewed tests and test coverage - [ ] manually tested (if applicable)
Description
Closes: #10705
State Sync for V2 Store (ADR-40)
Author Checklist
All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.
I have...
!
to the type prefix if API or client breaking changeCHANGELOG.md
Reviewers Checklist
All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.
I have...
!
in the type prefix if API or client breaking change