Periodically clean up cached bundles directory

[kostko]

Currently the cached exploded bundles are never removed which can result in a lot of disk use after a few upgrades. We should make it so that cached bundles are periodically removed if they are either:

• For versions lower than the active runtime version (regular bundles).
• From bundles that are no longer present in configuration (regular and detached bundles).

[martintomazic]

There might be exception for the paratime clients (?):

For versions lower than the active runtime version (regular bundles).

Haven't checked this in the code yet, but (see excerpt from the docs; updating paratimes):

If you are running a client node, ParaTime state syncing will be performed regardless of the version installed.

[kostko]

That is related to syncing runtime transactions and state. Since we are using state diffs, there is no need to have the correct runtime version as the runtime logic itself is not involved during storage sync.

[martintomazic]

cached bundles are periodically removed

I was thinking about keeping it simple i.e.:

• Single clean-up during/after newRuntimeConfig to handle stale exploded bundles no longer in the config.
• For versions lower then the current active one, doing a clean-up where the actual hosted runtime version changes (see) and also remove it from the hostConfig of the corresponding runtime registry?
  • Am I complicating and should simply watch WatchActiveDescriptor instead?
  • I am worried this might be bad idea if the runtime lags slighlty behind the consensus, or is this actually not possible by design?

Will hopefully have draft PR ready tmr :)

[kostko]

Single clean-up during/after

I would just make sure that this doesn't block initialization so while it should be fired on startup, it should be done in the background.

I am worried this might be bad idea if the runtime lags slighlty behind the consensus

What do you mean? To determine the active runtime version you only need to look at consensus (the active descriptor you mentioned). Runtimes and consensus are always in sync currently and once a version is deemed the active version, it can never revert to a version with a lower version number. So even if the node is behind, once a version is active in the consensus layer, you know that no earlier versions can ever activate for that runtime.