The benchmark suite is a highly flexible performance benchmarking framework, centered
around Shake build targets. However, it has high cognitive load for HLS non-experts. In
particular, it:
- Has multiple interconnected projects:
bench, ghcide-bench and shake-bench.
- Constructs an entire Shake build system
- Outputs large hierarchical CSV
- Has high levels of abstraction
That said, I propose the following changes:
- Consolidate
bench, ghcide-bench and shake-bench into a single bench project
- Completely remove
shake
- Replace orchestration/statistical processing with
tasty-bench, if reasonably fits
- Use familiar benchmarking terminology:
- Examples -> Projects or Harnesses
- Experiments -> Benchmarks
- Configurations -> separate into two distinct concepts:
- Test run configuration
- HLS plugin set
The benchmark suite is a highly flexible performance benchmarking framework, centered
around Shake build targets. However, it has high cognitive load for HLS non-experts. In
particular, it:
bench,ghcide-benchandshake-bench.That said, I propose the following changes:
bench,ghcide-benchandshake-benchinto a singlebenchprojectshaketasty-bench, if reasonably fits