Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When using the deleteVersion method, the disk size increases. #113

Closed
alexpath opened this issue Sep 13, 2018 · 4 comments
Closed

When using the deleteVersion method, the disk size increases. #113

alexpath opened this issue Sep 13, 2018 · 4 comments
Labels
T:bug Type: Bug (Confirmed)

Comments

@alexpath
Copy link

alexpath commented Sep 13, 2018

After each commit, call the deleteVersion function to delete the previous version.

The deletion attempted to reduce the size of the data, but the size increased.

Could you explain why this is happening?

Use version - v0.11.0

Compare size
version:10,000, size:3.1 MB (No delete previous version)
version:10,000, size:4.0 MB (Delete previous version)

@alexpath alexpath changed the title When use deleteVersion method When using the deleteVersion method, the disk size increases. Sep 13, 2018
@enlight
Copy link

enlight commented Oct 25, 2018

In my experience you'll need to compact the db after deleting old versions in order to actually reclaim disk space, leveldb has a CompactRange function you can call with an empty range to compact the whole db.

@ValarDragon
Copy link
Contributor

ValarDragon commented Oct 25, 2018

This seems troubling that deleting data is causing .9MB (at least) of io writes. The size not being reclaimed makes sense, more data being written doesn't seem like it should be the case. I think something is probably wrong here, either from a theoretic perspective of the IAVL schemes versioning, or just a sub-optimal implementation. I also suspect this same problem is likely a significant contributor to cosmos/cosmos-sdk#2131

@ValarDragon ValarDragon added T:bug Type: Bug (Confirmed) priority? and removed priority? labels Oct 25, 2018
@erikgrinaker
Copy link
Contributor

This seems troubling that deleting data is causing .9MB (at least) of io writes. The size not being reclaimed makes sense, more data being written doesn't seem like it should be the case.

This isn't necessarily true. LevelDB is log-structured, so deleting a key basically appends an additional log entry like:

some_random_key: delete

Every database key is a node hash and thus takes up about 40 byte each. It's also possible that orphan/inner node churn amplifies this.

@tac0turtle
Copy link
Member

this was removed in the last release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T:bug Type: Bug (Confirmed)
Projects
None yet
Development

No branches or pull requests

5 participants