Skip to content

feat: enhance JSON output for multi-profile validation #74

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

kikkomep
Copy link
Member

This PR refines the JSON output generated by validating a rocrate that requires multiple profiles, not belonging to the same inheritance hierarchy.

The main changes are as follows:

  • the output will be a single JSON object, unlike before when a separate JSON was produced for each validation profile used
  • the profile_identifier property is replaced by profile_identifiers, which will be populated with the list of validation profiles used
  • the issues property will contain the list of all issues encountered during the crate validation across the various involved validation profiles

@simleo
Copy link
Member

simleo commented Feb 25, 2025

I tried the following and got an error:

$ rocrate-validator -y validate -v -f json -o err.json -l REQUIRED tests/data/crates/valid/multi-profile-crate/
         This error may be due to a bug.
         Please report it to the issue tracker along with the following stack trace:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/simleo/git/rocrate-validator/rocrate_validator/cli/commands/validate.py:439 in validate    │
│                                                                                                  │
│   436 │   │   │   # Print the validation report to a file                                        │
│   437 │   │   │   if output_file:                                                                │
│   438 │   │   │   │   with open(output_file, "w") as f:                                          │
│ ❱ 439 │   │   │   │   │   f.write(json_output)                                                   │
│   440 │   │                                                                                      │
│   441 │   │   # using ctx.exit seems to raise an Exception that gets caught below,               │
│   442 │   │   # so we use sys.exit instead.                                                      │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: write() argument must be str, not dict


[ERROR] 

[FAILED] Unexpected error: write() argument must be str, not dict !!!

BTW, tests/data/crates/valid/multi-profile-crate does not seem to be valid with respect to https://w3id.org/ro/wftest/0.1, there is no test suite.

@simleo
Copy link
Member

simleo commented Feb 26, 2025

I ran:

rocrate-validator -y validate -v -f json -o err.json -l REQUIRED tests/data/crates/invalid/0_multi_profile_crate/

and got the attached err.json. profile_identifiers lists both workflow-testing-ro-crate-0.1 and provenance-run-crate-0.5, but no validation seems to have been performed with respect to workflow-testing-ro-crate-0.1: the fact that the root data entity does not refer to test suites via mentions (a MUST) is not reported.

Another weird thing is that if I set -p workflow-testing-ro-crate I only get issues if the output is in text format, while with JSON output I get this:

{
    "meta": {
        "version": "0.2"
    },
    "validation_settings": {
        "profile_identifier": "workflow-testing-ro-crate-0.1",
        "enable_profile_inheritance": true,
        "abort_on_first": false,
        "requirement_severity": "REQUIRED",
        "rocrate_validator_version": "0.6.1_bd0cc32+448-dirty"
    },
    "passed": false,
    "issues": []
}

@kikkomep kikkomep merged commit 079fed8 into crs4:develop Mar 7, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants