Skip to content

Commit

Permalink
store v2 doc
Browse files Browse the repository at this point in the history
  • Loading branch information
cool-develope committed Oct 3, 2024
1 parent 69914dc commit bde0726
Show file tree
Hide file tree
Showing 4 changed files with 129 additions and 12 deletions.
13 changes: 9 additions & 4 deletions store/v2/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Store

The `store` package contains the implementation of store/v2, which is the SDK's
abstraction around managing historical and committed state. See [ADR-065](../docs/architecture/adr-065-store-v2.md)
abstraction around managing historical and committed state. See [ADR-065](../../docs/architecture/adr-065-store-v2.md)
and [Store v2 Design](https://docs.google.com/document/d/1l6uXIjTPHOOWM5N4sUUmUfCZvePoa5SNfIEtmgvgQSU/edit#heading=h.nz8dqy6wa4g1) for a high-level overview of the design and rationale.

## Usage
Expand Down Expand Up @@ -42,21 +42,26 @@ sequenceDiagram
end
```

`Prune store keys` does not remove the data from the SC and SS instantly. It only
`PruneStoreKeys` does not remove the data from the SC and SS instantly. It only
marks the store keys as pruned. The actual data removal is done by the pruning
process of the underlying SS and SC.

## Migration

<!-- TODO -->
The migration from store/v1 to store/v2 is supported by the `MigrationManager` in
the `migration` package. See [Migration Manager](./migration/README.md) for more details.

## Pruning

The `root.Store` is NOT responsible for pruning. Rather, pruning is the responsibility
of the underlying SS and SC layers. This means pruning can be implementation specific,
such as being synchronous or asynchronous.
such as being synchronous or asynchronous. See [Pruning Manager](./pruning/README.md) for more details.


## State Sync

The `root.Store` is NOT responsible for state sync. See [Snapshots Manager](./snapshots/README.md)
for more details.

## Test Coverage

Expand Down
6 changes: 5 additions & 1 deletion store/v2/commitment/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,11 @@ See this [section](https://docs.google.com/document/d/1l6uXIjTPHOOWM5N4sUUmUfCZv

## Pruning

<!-- TODO -->
Pruning is the process of efficiently managing and removing outdated data from the
State Commitment (SC). To facilitate this, the SC backend must implement the `Pruner`
interface, allowing the `PruningManager` to execute data pruning operations according
to the specified `PruningOption`. Optionally, the SC backend can implement the
`PausablePruner` interface to pause pruning during a commit.

## State Sync

Expand Down
111 changes: 111 additions & 0 deletions store/v2/migration/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# Migration Manager

The `migration` package contains the `migration.Manager`, which is responsible
for migrating data from `store/v1` to `store/v2`. To ensure a smooth transition,
the process is designed to **lazily** migrate data in the background without blocking
`root.Store` operations.

## Overview

The migration process involves several steps:

1. **Create a snapshot** of the current state while `Commit` operations continue to
function with `store/v1`.
2. **Restore the snapshot** into the new StateStorage (SS) and StateCommitment (SC).
3. **Sync recent state changes** from `store/v1` to the new SS and SC.
4. After syncing, the `Commit` operation will be switched to the new `store/v2`.

Taking a snapshot is a lightweight operation. The snapshot is not stored on disk but
consumed by the `Restore` process, which replays state changes to the new SS and SC.

> **Note:** After migration, `store/v2` does **not** support historical queries.
If historical data access is required, a full state migration to `store/v2` is necessary.

## Usage

You can create a new `migration.Manager` by calling the following function:

```go
func NewManager(
db corestore.KVStoreWithBatch,
sm *snapshots.Manager,
ss *storage.StorageStore,
sc *commitment.CommitStore,
logger log.Logger
) *Manager
```

* `sc` (Commitment Store) can be `nil`. In that case, the Manager will migrate only
the state storage.
* The migration process is lazy, meaning data is migrated in the background while
`root.Store` remains fully operational.

To initiate the migration process, call the `Start` method:

```go
func (m *Manager) Start(ctx context.Context) error
```

> **Note:** It should be called by the RootStore, running in the background.

## Migration Flow

```mermaid
sequenceDiagram
autonumber

participant A as RootStore
participant B as MigrationManager
participant C as SnapshotsManager
participant D as StateCommitment
participant E as StateStorage

A->>B: Start
loop Old Data Migration
B->>C: Create Snapshot
C->>B: Stream Snapshot
B->>D: State Sync (Restore)
B->>E: Write Changeset (Restore)
end

loop New Commit Data Sync
A->>B: Commit(Changeset)
B->>B: Store Changeset
B->>D: Commit Changeset
B->>E: Write Changeset
end

B->>A: Switch to new store/v2
```

## Key Considerations

### Laziness and Background Operation

The migration is performed lazily, meaning it occurs in the background without
interrupting the current operations on root.Store. This allows the chain to continue
running while data is gradually migrated to `store/v2`. State synchronization ensures
that any new state changes during the migration are also applied to `store/v2`.

However, note that there may be a performance impact depending on the size of the data
being migrated, and it’s essential to monitor the migration process in production
environments.

### Handling Failures and Rollbacks

It is important to consider how the migration manager handles errors or system failures
during the migration process:

* If the migration fails, there is no impact on the existing `store/v1` operations,
but need to restart the migration process from the scratch.
* In the event of a critical failure after migration, a rollback may not be possible,
and it is needed to keep the `store/v1` backup for a certain period.

### Impact on Historical Queries

After the migration, the new `store/v2` does not support historical queries.
This limitation should be clearly understood before starting the migration process,
especially if the node relies on historical data for any operations.

If historical queries are required, users must fully migrate all historical data to `store/v2`.
Alternatively, keeping store/v1 accessible for historical queries could be an option.
11 changes: 4 additions & 7 deletions store/v2/storage/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,13 +70,10 @@ Iterate/backend_rocksdb_versiondb_opts-10 778ms ± 0%

## Pruning

Pruning is an implementation and responsibility of the underlying SS backend.
Specifically, the `StorageStore` accepts `store.PruningOption` which defines the
pruning configuration. During `ApplyChangeset`, the `StorageStore` will check if
pruning should occur based on the current height being committed. If so, it will
delegate a `Prune` call on the underlying SS backend, which can be defined specific
to the implementation, e.g. asynchronous or synchronous.

Pruning is the process of efficiently managing and removing outdated or redundant
data from the State Storage (SS). To facilitate this, the SS backend must implement
the `Pruner` interface, allowing the `PruningManager` to execute data pruning operations
according to the specified `PruningOption`.

## State Sync

Expand Down

0 comments on commit bde0726

Please sign in to comment.