store/rootmulti: On initialization, prune all old versions according to existing settings

[ValarDragon]

Summary

If you update pruning settings, it will never prune old state. Prune heights are tracked by a pruneHeights variable in RAM thats updated after each block, and a separate region of disk being written to track recent blocks that need to be pruned at next pruning interval.

However, at rootmulti store load, it should get all versions from the underlying database (or at least the oldest version to handle removing keep-every), and then prune everything that should be pruned according to the new settings.

Problem Definition

So node operators can change their pruning settings, and actually get space savings.

Proposal

• Add a GetAllVersions function to IAVL's nodeDB.

• In RootMultiStore, change the following logic in loadVersion: https://github.com/cosmos/cosmos-sdk/blob/master/store/rootmulti/store.go#L265-L269 to the following (pseudocode):

func (rs *Store) PruneOnInitialLoad() {
  if rs.HasAlreadyPrunedThisLoad || rs.pruningOptions.KeepEvery == 0 {
     return
  }

	for key, store := range rs.stores {
		if store.GetStoreType() == types.StoreTypeIAVL {
			// If the store is wrapped with an inter-block cache, we must first unwrap
			// it to get the underlying IAVL store.
			store = rs.GetCommitKVStore(key)

                        latestVersion := store.GetLatestVersion()
                        allVersions := store.GetAllVersions()
                        pruneVersions := []int64{}
                        for v := range allVersions {
                            if v % rs.PruningOptions.KeepEvery != 0 && v + rs.PruningOptions.KeepRecent < latestVersion {
                                pruneVersions = append(pruneVersions, v)
  }
}

			if err := store.(*iavl.Store).DeleteVersions(pruneVersions...); err != nil {
				if errCause := errors.Cause(err); errCause != nil && errCause != iavltree.ErrVersionDoesNotExist {
					panic(err)
				}
			}
		}
	}
}

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned

[tac0turtle]

Is this urgent or could we get it in with the work you are doing on IAVL?

[ValarDragon]

Its not really urgent, just means that you can't change pruning settings to get space savings on a current node.

On chains that don't have state sync (all cosmwasm chains), theres not really a good way to lower disk size in the case of bad pruning configuration.

[tac0turtle]

another possibly simpler approach could be to create an inverted function to DeleteVersionsFrom that deletes up to said version instead of getting all versions then calling delete on them.

I would love to get this sooner rather than later cause of the rate of growth on networks.

[yihuang]

It seems easy enough to implement a cli cmd to prune old states according to current settings?

[alexanderbez]

Yes, a CLI command should exist to manually prune. Would love to see this work <3

[elias-orijtech]

With #12742 merged is this issue still relevant?

[tac0turtle]

I believe with the upcoming iavl changes this should be solved as well. In the new change we prune up to height so if you make it shorter it will prune all things in the past

Closing this due to the new api