perf: opossum

[H4ad]

Inspired by https://twitter.com/4Kschool/status/1737218913212170699, I tried to search for performance improvements in this library.

perf(circuit): use bitset instead of storing valeus inside circuit

Basically, with this change, I tried to condense and reduce the amount of fields/memory needed to store the same information.

perf(circuit): remove amount of listeners

Instead of incrementing the metrics using listeners, I directly increase the number together when I emit the event.
This can introduce a little bit of maintenance burden but also save some memory and increase a little bit the performance for .emit since they didn't need to call a function.

perf(semaphore): reduce memory footprint for semaphore

With this rewrite, I reduce the amount of memory needed and this also improves the instantation time of this class.

refactor(semaphore): removed unused timeout

Was not used by this library, so I removed it, let me know if you plan something with this feature and I can revert this commit.

perf(circuit): avoid cloning args

Now we have a specialized function that receives the array and we don't need anymore to shallow-clone the args twice.

perf(circuit): avoid calling errorFilter when not needed

Just a minor perf improvement, avoid calling a function that we know the result.

perf(circuit): avoid args cloning on fallback fn

This is a little bit more controversial, I didn't see any reason to avoid modifying the args array, so this can save a little bit of memory by avoiding this shallow-clone.

perf(status): avoid creating array unecessary for bucket keys

Trying to reduce the amount of memory allocated by caching the keys instead of capturing those keys every time.

perf(status): prefer modify original array instead concat old arrays

There's no reason to avoid modifying that array since it was created during .reduce, so this helps the performance a little bit and also saves a little bit of memory.

perf(semaphore): avoid promise creation when is not needed

A simple change that speeds a little bit the fire action since we avoid creating an unnecessary promise.

---

In general, I tried to measure the perf improvements and using bench-node I got the following results:

Before:

create empty circuit x 197,114 ops/sec +/- 0.07% (7 runs sampled) heap usage=3.40Kb (3.30Kb ... 3.49Kb) min..max=(4.68us ... 5.68us) p75=5.18us p99=5.68us
create circuit x 178,761 ops/sec +/- 0.02% (4 runs sampled) heap usage=3.45Kb (3.38Kb ... 3.59Kb) min..max=(5.48us ... 5.72us) p75=5.58us p99=5.72us
create semaphore x 1,331,163 ops/sec +/- 0.2% (2 runs sampled) heap usage=419B (384B ... 454B) min..max=(606ns ... 809ns) p75=809ns p99=809ns
simple fire x 942,144 ops/sec +/- 0.67% (5 runs sampled) heap usage=151B (33B ... 295B) min..max=(996ns ... 3.30us) p75=1.14us p99=3.30us
failure fire x 210,998 ops/sec +/- 0.07% (5 runs sampled) heap usage=1.12Kb (795B ... 1.64Kb) min..max=(4.09us ... 4.96us) p75=4.74us p99=4.96us

create empty circuit x 189,293 ops/sec +/- 0.21% (8 runs sampled) heap usage=3.43Kb (3.34Kb ... 3.48Kb) min..max=(4.52us ... 8.07us) p75=6.01us p99=8.07us
create circuit x 176,277 ops/sec +/- 0.1% (5 runs sampled) heap usage=3.41Kb (3.35Kb ... 3.44Kb) min..max=(5.20us ... 6.62us) p75=5.61us p99=6.62us
create semaphore x 1,366,766 ops/sec +/- 0.23% (2 runs sampled) heap usage=349B (311B ... 387B) min..max=(584ns ... 806ns) p75=806ns p99=806ns
simple fire x 944,067 ops/sec +/- 0.64% (5 runs sampled) heap usage=131B (58B ... 281B) min..max=(964ns ... 3.18us) p75=1.23us p99=3.18us
failure fire x 203,585 ops/sec +/- 0.02% (4 runs sampled) heap usage=1020B (760B ... 1.64Kb) min..max=(4.75us ... 5.00us) p75=4.98us p99=5.00us

create empty circuit x 190,282 ops/sec +/- 0.05% (6 runs sampled) heap usage=3.42Kb (3.35Kb ... 3.46Kb) min..max=(5.14us ... 5.86us) p75=5.27us p99=5.86us
create circuit x 173,088 ops/sec +/- 0.05% (4 runs sampled) heap usage=3.42Kb (3.41Kb ... 3.44Kb) min..max=(5.36us ... 5.96us) p75=5.95us p99=5.96us
create semaphore x 1,202,554 ops/sec +/- 0.18% (2 runs sampled) heap usage=409B (375B ... 443B) min..max=(735ns ... 953ns) p75=953ns p99=953ns
simple fire x 957,888 ops/sec +/- 0.5% (5 runs sampled) heap usage=167B (37B ... 447B) min..max=(983ns ... 2.53us) p75=1.14us p99=2.53us
failure fire x 204,377 ops/sec +/- 0.03% (4 runs sampled) heap usage=1.01Kb (792B ... 1.64Kb) min..max=(4.64us ... 4.98us) p75=4.89us p99=4.98us

After:

create empty circuit x 253,026 ops/sec +/- 0.14% (7 runs sampled) heap usage=3.04Kb (3.03Kb ... 3.06Kb) min..max=(3.60us ... 5.25us) p75=4.36us p99=5.25us
create circuit x 234,540 ops/sec +/- 0.08% (3 runs sampled) heap usage=3.05Kb (3.04Kb ... 3.06Kb) min..max=(3.93us ... 4.59us) p75=4.59us p99=4.59us
create semaphore x 431,598,754 ops/sec +/- 1.25% (2 runs sampled) heap usage=42B (3B ... 81B) min..max=(1.47ns ... 24.27ns) p75=24.27ns p99=24.27ns
simple fire x 1,076,134 ops/sec +/- 0.66% (4 runs sampled) heap usage=176B (45B ... 472B) min..max=(880ns ... 2.74us) p75=982ns p99=2.74us
failure fire x 216,585 ops/sec +/- 0.01% (3 runs sampled) heap usage=1.04Kb (802B ... 1.56Kb) min..max=(4.59us ... 4.64us) p75=4.64us p99=4.64us

create empty circuit x 262,022 ops/sec +/- 0.1% (8 runs sampled) heap usage=3.05Kb (3.02Kb ... 3.08Kb) min..max=(3.60us ... 4.72us) p75=4.04us p99=4.72us
create circuit x 230,788 ops/sec +/- 0.02% (4 runs sampled) heap usage=3.07Kb (3.05Kb ... 3.10Kb) min..max=(4.21us ... 4.46us) p75=4.38us p99=4.46us
create semaphore x 302,207,559 ops/sec +/- 1.25% (2 runs sampled) heap usage=41B (3B ... 78B) min..max=(1.93ns ... 31.10ns) p75=31.10ns p99=31.10ns
simple fire x 919,432 ops/sec +/- 0.53% (4 runs sampled) heap usage=212B (103B ... 433B) min..max=(965ns ... 2.62us) p75=1.22us p99=2.62us
failure fire x 211,607 ops/sec +/- 0.08% (4 runs sampled) heap usage=991B (778B ... 1.56Kb) min..max=(4.04us ... 4.97us) p75=4.70us p99=4.97us

create empty circuit x 267,593 ops/sec +/- 0.14% (7 runs sampled) heap usage=3.04Kb (3.02Kb ... 3.06Kb) min..max=(3.36us ... 4.96us) p75=4.14us p99=4.96us
create circuit x 230,268 ops/sec +/- 0.05% (4 runs sampled) heap usage=3.05Kb (3.03Kb ... 3.07Kb) min..max=(4.08us ... 4.66us) p75=4.42us p99=4.66us
create semaphore x 461,537,772 ops/sec +/- 1.28% (2 runs sampled) heap usage=50B (3B ... 96B) min..max=(1.32ns ... 25.79ns) p75=25.79ns p99=25.79ns
simple fire x 966,567 ops/sec +/- 0.6% (3 runs sampled) heap usage=224B (84B ... 390B) min..max=(975ns ... 2.60us) p75=2.60us p99=2.60us
failure fire x 209,872 ops/sec +/- 0.05% (4 runs sampled) heap usage=986B (747B ... 1.56Kb) min..max=(4.57us ... 5.09us) p75=4.73us p99=5.09us

benchmark code

const CircuitBreaker = require('../index');
const Semaphore = require('../lib/semaphore');
const { Suite, MemoryEnricher } = require('bench-node');

async function asyncFunctionThatCouldFail (x, y) {
}

const breaker = new CircuitBreaker(asyncFunctionThatCouldFail, {
  timeout: 3000, // If our function takes longer than 3 seconds, trigger a failure
  errorThresholdPercentage: 50, // When 50% of requests fail, trip the circuit
  resetTimeout: 30000, // After 30 seconds, try again.
});


const failureBreaker = new CircuitBreaker(async (r) => {
  throw new Error('err');
}, {
  timeout: 3000, // If our function takes longer than 3 seconds, trigger a failure
  errorThresholdPercentage: 50, // When 50% of requests fail, trip the circuit
  resetTimeout: 30000, // After 30 seconds, try again.
});

const benchOptions = {
  maxTime: 2,
  minTime: 0.2
};

const suite = new Suite()
  .add('create empty circuit', () => new CircuitBreaker(asyncFunctionThatCouldFail), benchOptions)
  .add('create circuit', () => new CircuitBreaker(asyncFunctionThatCouldFail, {
    timeout: 3000, // If our function takes longer than 3 seconds, trigger a failure
    errorThresholdPercentage: 50, // When 50% of requests fail, trip the circuit
    resetTimeout: 30000, // After 30 seconds, try again.
  }), benchOptions)
  .add('create semaphore', () => new Semaphore(2))
  .add('simple fire', async () => await breaker.fire(1, 2), benchOptions)
  .add('failure fire', async () => {
    try {
      await failureBreaker.fire(1);
    } catch (e) {

    }
  }, benchOptions)
  .run({
    enrichers: [
      MemoryEnricher,
    ]
  });

I also tried to measure the overhead of simple fire using the following code:

hyperfine --warmup 3 'node perf-fire.cjs'

const CircuitBreaker = require('../index');

async function asyncFunctionThatCouldFail (x, y) {
}

const breaker = new CircuitBreaker(asyncFunctionThatCouldFail, {
  timeout: 3000, // If our function takes longer than 3 seconds, trigger a failure
  errorThresholdPercentage: 50, // When 50% of requests fail, trip the circuit
  resetTimeout: 30000, // After 30 seconds, try again.
});

(async () => {
  for (let i = 0; i < 1e6; i++) {
    await breaker.fire();
  }
})();

The output using hyperfine is:

Before:

Benchmark 1: node perf-fire.cjs                                                                                                            
  Time (mean ± σ):     915.4 ms ±  35.9 ms    [User: 936.0 ms, System: 49.4 ms]                                                            
  Range (min … max):   861.6 ms … 983.8 ms    10 runs                                                                                      

After:

Benchmark 1: node perf-fire.cjs
  Time (mean ± σ):     771.1 ms ±  10.2 ms    [User: 788.3 ms, System: 52.5 ms]
  Range (min … max):   748.0 ms … 783.7 ms    10 runs

[github-actions]

This pull request is stale because it has been open 30 days with no activity.

[H4ad]

Hey @lholmquist, can you take a look at this PR?

If you don't think it's worth those changes, let me know to close this one.