Adds callgrind as additional benchmarking harness. #12811
+1,348
−3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
FilipAndersson245
changed the title
fdncred
changed the title
fdncred
changed the title
sholderbach
reviewed
May 16, 2024
Comment on lines
+1
to
+5
| [ | ||
| { | ||
| "_id": "6623c00aa16be249c931db66", | ||
| "index": 0, | ||
| "guid": "67b63043-00ae-4dfc-9082-f1a994af30f3", |
There was a problem hiding this comment.
What is the provenance of this example file? Just to make sure we don't run into any license issues with it.
There was a problem hiding this comment.
I am unable to find the source of the data as it was a few weeks ago, I took it from some example JSON site as I thought it looked quite realistic to how people use JSON. I'm unaware if there were any licenses on the data.
| @@ -232,6 +232,7 @@ pretty_assertions = { workspace = true } | |||
| rstest = { workspace = true, default-features = false } | |||
| serial_test = "3.1" | |||
| tempfile = { workspace = true } | |||
| iai-callgrind = "0.10.2" | |||
There was a problem hiding this comment.
With the limits on only having dev-dependencies, how will this impact the test build times?
There was a problem hiding this comment.
comparing the timings from cargo test --no-run --timings
main: 28.5s
this-PR: 29.5s
FilipAndersson245
force-pushed
the
callgrind-benches
branch
from
991daef
to
d2ccc71
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This is a test in addition to a time-based measurement we would also be able to use callgrid, It runs on simplified software CPU to calculate metrics of instructions, L1, L2, and RAM access. It is not 100% representative to a real CPU, but can most of the time be a very good heuristic. SQLite uses it as its default benchmarking tool.
I created a few bench cases that instead of running the internal library part of nushell, instead run the binary directly hopefully allowing better and simpler implementations of future benchmarks.
User-Facing Changes
Tests + Formatting
After Submitting