Generator of compliance tests for OPA IR evaluators. Consumes OPA yaml test-cases, and produces a corresponding set of json test-cases that includes compiled IR plans.
- Initialize/update the OPA submodule
- In the project root:
git submodule update --init
- (optional) Check out wanted OPA version:
cd opa && git checkout <commit|branch|tag>
- In the project root:
go mod download
go run . <TARGET_PATH>
On completion, TARGET_PATH
will contain a set of folders, roughly corresponding to test categories. Each folder will contain a set of .json
files, with the following structure:
cases
(list of dictionaries): a list of test-case definitionsnote
(string): the name of the test-caseplan
(dictionary): the compiled IR for the test-caseentrypoints
(list of string): a list of entrypoints in the IR to executedata
(dictionary): thedata
document to execute each entrypoint withinput
(dictionary): theinput
document to execute each entrypoint withwant_plan_result
(dictionary): a dictionary with the expected output, mapped by entrypoint name. (mutually exclusive withwant_error_code
andwant_error
)want_error_code
(string): if present, the error code of an expected error (mutually exclusive withwant_plan_result
)want_error
(string): if present, the error message of an expected error (mutually exclusive withwant_plan_result
)strict_error
(boolean): if present, errors generated by built-in functions should not produce anundefined
result, instead, execution should be halted with that error.
Note: when an error is expected, the test-case definition doesn't indicate which entrypoint
will spawn it. Therefore, when multiple entrypoints are present, all must be executed, but not all may spawn the expected error (but at least one must).