Alternative serialization formats

[albe]

NOTE: This is just an informational issue and not actionable/not supposed to create a change

By default we use native JSON.stringify/JSON.parse for serializing events for storage. In my benchmarks with msgpack and protobuf JSON outperformed those by an order of magnitude. The events are human readable in the file (though the additional metadata added lately somewhat convolutes the readability), but filesize is bigger than with one of the other formats.

Nodejs also provides a native serialization format via v8 (https://nodejs.org/api/v8.html#v8_serialization_api), but in a benchmark test (nodejs 15.8) that also turned out an order of magnitude slower than JSON:

const document = { type: 'FooBarHappened', payload: { foo: 'foo', bar: 'bar', baz: 'baz', quux: 'quux', price: 25.74, amount: 76492 }, metadata: { version: 0, occuredAt: Date.now(), commitVersion: 0 } };

Suite.add('JSON', () => {
    document.metadata.version++;
    JSON.parse(JSON.stringify(document));
});
Suite.add('v8.serialize', () => {
    document.metadata.version++;
    v8.deserialize(v8.serialize(document));
});

  JSON         x 272,929 ops/sec ±1.79% (85 runs sampled)
  v8.serialize x  27,068 ops/sec ±25.31% (39 runs sampled)


  JSON.stringify x 457,605 ops/sec ±2.80% (81 runs sampled)
  v8.serialize   x  40,960 ops/sec ±30.39% (32 runs sampled)

  JSON.parse     x 521,375 ops/sec ±3.51% (78 runs sampled)
  v8.deserialize x 250,910 ops/sec ±2.76% (78 runs sampled)

For another benchmark of multiple serialization methods:
https://github.com/Adelost/javascript-serialization-benchmark

avro seems promising, but requires a schema for the serialized documents.