feat(gatsby): node persistence

[vladar]

Description

Currently gatsby stores all data in NodeJS heap and persists it between builds using v8.serialize. This causes significant memory pressure when there are too many nodes, especially during the persistence step.

This PR adds another experimental data storage option: lmdb-store. Instead of keeping nodes in memory, they are instantly saved to this persistent embeddable storage. So there is no need to have a separate persistence step.

It imposes some additional constraints to plugins and gatsby sites, namely:

1. Circular references in nodes are not allowed (will throw when trying to save)
2. Node mutations are not allowed (e.g. assigning NODE___featuredImage after a node was created)
3. Others - to be discovered

Note: this PR only moves nodes and nodesByType to LMDB. The rest of the build state remains in redux cache and persisted as usual.

This is the first PR in the series as LMDB unblocks several potential performance improvements. But currently, it may actually cause some memory overhead (average memory usage increases but peak memory spikes decrease, so fewer OOMs) + 10-15% slowdown during sourcing. Eventually, we will cut those down but that's the first step.

Try it

1. Make sure you use Node 14.10+
2. Set env variable GATSBY_EXPERIMENTAL_LMDB_STORE=1 and run gatsby build (or develop) or alternatively, enable it in gatsby-config.js:

module.exports = {
  flags: {
    LMDB_STORE: true
  }
}

3. yarn add lmdb-store (since it is an experimental feature, lmdb-store is not a dependency yet and you must add it to your site explicitly)