Fix bug in test.sh
counterexample error matcher 🐞
#288
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Category
What kind of change is this?
Please select one of the following four options.
Consult Pull request merging criteria for a description of each category.
Description
Brief description of the business purpose and effect of the pull request.
This commit fixes a bug in the
test.sh
script that made it give false positive test passes on all counterexamples. It then fixes all the counterexamples to ensure they pass with the fixed script. 😰The genesis of the
test.sh
bug was the output changes in thejv
tool. We've been working to fix our testing setup in response to thejv
changes since around June 9, 2024 with commit8071ea8b
, but there's been a bit of whack-a-mole as fixes for problems beget new problems. Hopefully this is the last one in the series!Reference
List of relevant links to GitHub issues, PRs, and other documentation.
test.sh
#214Testing
Brief description of the testing done for this change showing why you are confident it works as expected and does not introduce regressions. Provide sample output data where appropriate.
test.sh
and found they were almost all broken.Checklist
Checklist of tasks commonly-associated with schema pull requests. Please review the relevant checklists and ensure you do all the tasks that are required for the change you made.
A
but is not intended to test propertyA
's validity, and you made a schema change that invalidates propertyA
in that counterexample, fix the counterexample to align it with your schema change.Documentation Website
Update the hyperlink below to put the pull request number in.
Docs preview for this PR.