Nightly Valgrind Matrix

[reneme]

This sets up a configuration matrix for the nightly Valgrind run. In the hope to generate a stronger signal from the CT::poison() helpers. For that, I introduced two new build targets 'valgrind-ct' and 'valgrind-ct-full'. Currently, as their only difference compared to the non-ct targets, they don't enable the --leak-check=full, --show-reachables and --track-origins Valgrind features.

Explicitly note that the clang -Os variant would have been able to detect #4107, as evident by the ct_selftest.py test "clang_vs_bare_metal_ct_mask... ok" which reproduced this issue. This particular self-test is skipped for all other configurations.

Below are some runtimes. Note that the compiliation results were cached in ccache (from previous test runs). So these runtimes are mostly comprised of runner setup, cached compilation and valgrind-hosted test suite runtime.

	GCC	clang
-O1	52min	63min
-O2	54min	59min
-O3	49min	50min
-Os	63min 🔸	59min

🔸, means that it ran a reduced set of tests (valgrind-ct)

The valgrind configuration matrix also contains the existing 'valgrind-full' run on clang with -O3. This is the only configuration that explicitly aims to find memory bugs and enables the --leak-check=full, --show-reachables and --track-origins Valgrind features.

---

Here are, now outdated, runtimes (before switching off --leak-check, --show-reachables, --track-origins) and compiling with a cold ccache:

	GCC	clang
-O0	❌	❌
-O1	1h35m	1h43m
-O2	1h34m	1h33m
-O3	1h30m	1h36m
-Os	2h16m 🔸	1h50m

❌, means that they failed in a timeout after 6 hours.
🔸, means that it timed out with valgrind-full but succeeded with valgrind

Closes #4396

[randombit]

I think the no optimizations case is not really that useful; if someone using using that in production they are insane since performance will be terrible, we quite explicitly rely on the compiler performing extensive inlining/etc.

Same logic applies to -O1, but there the performance seems tolerable (considering).

There are two completely distinct reasons we're running valgrind

• Detecting memory errors/leaks, the normal valgrind stuff
• To detect const time violations

We should distinguish these in the build

• valgrind runs all tests, with our current set of valgrind flags (--leak-check=full, --show-reachable, --track-origins - recall that these add significant overhead to running valgrind), and using our standard compilation flags.
• valgrind-ct (whatever) runs only specifically designated tests, and without the leak check/origin tracking flags, across a range of targets.

[reneme]

I think the no optimizations case is not really that useful; if someone using that in production they are insane since performance will be terrible.

I wanted to agree, but the -O0 did actually find something that may or may not be worth fixing: 67964a8

[reneme]

valgrind-ct (whatever) runs only specifically designated tests, and without the leak check/origin tracking flags, across a range of targets.

Given that the runtime is bearable (for a nightly job and excluding -O0), I am wondering whether it is really worth the effort of splitting these out. Perhaps to tweak the valgrind flags for the CT-matrix. But I'd put a question mark on the hand-picked tests, as we'll inevitably miss adding relevant things to this list in the future.

[coveralls]

coverage: 91.073%. remained the same
when pulling cdd6fc3 on Rohde-Schwarz:ci/valgrind_matrix
into 2ae990f on randombit:master.

[randombit]

Looks good thanks