Rewrite to N-API

[vweevers]

Closes #100. Though this looks like a big PR, there's not much code to review because I copied large parts from leveldown. That includes binding.cc and all JS files including tests (rocksdb has only one test of its own: open-read-only-test.js). The differences between leveldown and rocksdb are recorded in 3433541, 7a5547b and bcdd317.

Prebuilds

Like leveldown, rocksdb now uses prebuildify which means prebuilds will be included in the npm package. Unlike leveldown, rocksdb does not make prebuilds for ARM, Android or Alpine (until someone requests them, I figured) (and on RPi and mobile you're better off with RocksDBLite). After initial review of this PR I'll trigger CI to make prebuilds, so we can check what the total npm package size will be.

Additional changes

• Disable block cache if cacheSize is 0, rather than using a block cache with RocksDB's default size
• Set verify_checksums to false (same as leveldown / LevelDB) (Support checksum options leveldown#172 (comment))
• Enable bloom filter (same as leveldown / LevelDB)
• Increase info_log_level from INFO_LEVEL to WARN_LEVEL

Performance

The leak tests (node --expose-gc test/leak-tester[-batch]) show memory growth but I get the same results on master (I used node 10.14.1 on both) so it might be a preexisting leak or natural RocksDB growth.

I want to run benchmarks as well; perhaps by extracting the benchmarks from leveldown into a standalone tool that can target any abstract-leveldown implementation.

[ralphtheninja]

Nice work. LGTM!

[vweevers]

Source code alone, package size is 1.8 MB, 8.5 MB unpacked. With prebuilds, the package size is 5.9 MB, 19.5 MB unpacked.

npm notice 3.5MB   prebuilds/darwin-x64/node.napi.glibc.node
npm notice 3.5MB   prebuilds/linux-x64/node.napi.glibc.node
npm notice 3.9MB   prebuilds/win32-x64/node.napi.glibc.node

(side note: I should remove removed the libc tag from the windows prebuild) (no harm, just looks weird)

[vweevers]

(first test of benchmark code, conditions may not have been optimal)

[vweevers]

Interesting: this branch of rocksdb is faster (on writes) than master, which is slower than leveldown:

I'll try to finish those benchmarks over the weekend so y'all can play too 😄

[vweevers]

Benchmark code available here.

With some npm magic you can install rocksdb@3 and the napi master branch side-by-side and compare their performance. Requires npm >= 6.9.0 (for aliases) but also node < 12 (for rocksdb@3). If you can't upgrade npm then use npx npm@latest:

mkdir tmp && cd tmp

npx npm@latest i rocksdb3@npm:rocksdb@3 Level/rocksdb --no-save
npm i level-bench -g

level-bench run write rocksdb3
level-bench run write rocksdb
level-bench plot write

Repeat the last 3 steps for the other benchmarks (write-random and write-sorted). You may need to reduce the amount of data for these benchmarks (gnuplot crashed on me). You can do so with -b [-n 1e6], for example:

level-bench run write-sorted rocksdb3 -b [-n 1e6]
level-bench run write-sorted rocksdb -b [-n 1e6]
level-bench plot write-sorted

Last note: you don't have to npm-install rocksdb, you can also clone the repo, install as usual, and run level-bench run <benchmark> (without further arguments) in that working directory.

Results on Windows (TLDR: we're good, though we don't have read benchmarks yet):

[vweevers]

More benchmarks are welcome, post results here! In the mean time I'm merging this.