Introduce benchmark + performance regression checks

[Ocramius]

As Psl\ grows in size and adoption, its performance becomes one important evaluation point when considering it for downstream adoption.

This question came up in both #194 and Roave/BackwardCompatibilityCheck#306, and it would be a good idea to have reference/comparison.

The idea is to have following benchmarks:

• comparison of Psl and internal functions or constructs it replaces (assuming 100% type coverage and no runtime errors occur)
• comparison of Psl against itself at different releases

For that, it would be interesting to use https://github.com/phpbench/phpbench/tree/1.0.1

@dantleech do you know if there is a sensible way to run benchmarks and compare them in CI? That's something that is kinda missing within PHPBench' docs 🤔

[azjezz]

some functions will always be slower than core ( e.g: Regex\matches will always perform worst than preg_match, due to additional error handling ), the question is how much difference is considered acceptable.

[Ocramius]

Obviously, Psl will always be slower than core: it is required to keep the baseline under control though, as performance regressions often go unnoticed.

Also, knowing how much slower the library is allows for downstream consumers to take an informed decision as to when/where to use it.

[dantleech]

r.e. running in CI - I'm doing this with a bash script with assertions at a tolerance of 10% (but this is too optimistic, 15% might be better). It's still an experimental area (i also tried two separate actions to combine their results, but this doesn't work as the actions can run in completely different envs I guess).

[Ocramius]

Sounds feasible - even with a very high tolerance, it's more about looking at it before releasing, I guess?

Intra-worker performance should be relatively stable

[dantleech]

it's more about looking at it before releasing, I guess?

with a 15% tolerance it should catch obvious regressions, I guess if you want to check more subtly then before release/at PR time.

Intra-worker performance should be relatively stable

as in 2 actions? at least for me (the one time i tried it) this resulted in less stable results

[Ocramius]

as in 2 actions? at least for me (the one time i tried it) this resulted in less stable results

No no, same action and container: nothing guarantees that you will get two boxes running on a similar hardware otherwise.