v0.11.0

v0.11 of BlinkDB arrives with massive improvements to the query engine, specifically for limiting & sorting results.

Before

blinkdb\limit-from.ts --- lokijs is 5.00x faster than blinkdb
┌─────────┬───────────┬───────────────────┬───────────────────┬──────────┬─────────┐
│ (index) │   name    │      ops/sec      │ Average Time (ns) │  Margin  │ Samples │
├─────────┼───────────┼───────────────────┼───────────────────┼──────────┼─────────┤
│    0    │ 'lokijs'  │ 16588.13217232204 │ 60284.06270288466 │ '±1.01%' │  8295   │
│    1    │ 'blinkdb' │ 3319.452529365677 │ 301254.4963825986 │ '±3.59%' │  1668   │
└─────────┴───────────┴───────────────────┴───────────────────┴──────────┴─────────┘
blinkdb\sort-id-asc.ts --- lokijs is 1.22x faster than blinkdb
┌─────────┬───────────┬────────────────────┬───────────────────┬──────────┬─────────┐
│ (index) │   name    │      ops/sec       │ Average Time (ns) │  Margin  │ Samples │
├─────────┼───────────┼────────────────────┼───────────────────┼──────────┼─────────┤
│    0    │ 'lokijs'  │ 1882.3348864078623 │ 531255.0955841558 │ '±2.41%' │   942   │
│    1    │ 'blinkdb' │ 1541.504252178804  │ 648716.9909434716 │ '±7.26%' │   771   │
└─────────┴───────────┴────────────────────┴───────────────────┴──────────┴─────────┘

After

blinkdb\limit-from.ts --- blinkdb is 3.10x faster than lokijs
┌─────────┬───────────┬────────────────────┬────────────────────┬──────────┬─────────┐
│ (index) │   name    │      ops/sec       │ Average Time (ns)  │  Margin  │ Samples │
├─────────┼───────────┼────────────────────┼────────────────────┼──────────┼─────────┤
│    0    │ 'blinkdb' │ 50585.49415326716  │ 19768.513024111937 │ '±8.57%' │  25293  │
│    1    │ 'lokijs'  │ 16296.938372072096 │  61361.218725211   │ '±2.90%' │  8156   │
└─────────┴───────────┴────────────────────┴────────────────────┴──────────┴─────────┘
blinkdb\sort-id-asc.ts --- blinkdb is 2.61x faster than lokijs
┌─────────┬───────────┬────────────────────┬────────────────────┬──────────┬─────────┐
│ (index) │   name    │      ops/sec       │ Average Time (ns)  │  Margin  │ Samples │
├─────────┼───────────┼────────────────────┼────────────────────┼──────────┼─────────┤
│    0    │ 'blinkdb' │ 4057.786560018582  │ 246439.77331213318 │ '±7.94%' │  2029   │
│    1    │ 'lokijs'  │ 1557.6211865036469 │ 642004.6213191763  │ '±2.88%' │   779   │
└─────────┴───────────┴────────────────────┴────────────────────┴──────────┴─────────┘

Implementation details

BlinkDB achieves this massive speedup by exploiting certain edge cases in queries:

• In previous implementations, limiting results with from was performed after all items were retrieved from the underlying btree. from is now respected directly in the btree in all cases where where the query engine uses the primary column for item retrieval.
• Sorting items can be entirely skipped if the query engine retrieved items from the primary column, since the Btree already returns the items sorted.