[Discussion]: Evaluation scoring: unclear how to score non-mandatory items

[manics]

Summary

How should optional and recommended items be scored?

Source

Discussion in #272

Detail

The current instructions are:

You should score your TRE against each statement in the SATRE specification using this scoring system:

:0 Not met: The TRE does not meet this requirement (the TRE is not SATRE compliant)
:1 Sufficient: The TRE meets this requirement met but there is substantial scope for improvement
:2 Satisfied: The TRE meets this requirement met but there may still be scope for improvement

A score of **1** or above means you have met the requirement.
Optionally you can use **1** and **2** to indicate potential areas of improvement in your TRE.

If you don't do implement an optional or recommended item should you score yourself 1 since you're still compliant with SATRE even though you don't do it, 0 since you don't do it (though according to the text this means you're not SATRE compliant), 2 since you've done the maximum you need/plan to (even if that's nothing), or N/A to clearly indicate it's not applicable?

Intended Output

Clarity on scoring items that aren't mandatory. This is particularly important for conditional statements such as those in
#272 (comment)

Who can help

No response

[JimMadge]

For scoring, I think either 2 (meaning, we don't need to do this so couldn't possibly improve) or N/A would work. N/A more clearly shows that the point is not relevant to the assessors TRE at a quick glance.

Adding the asterisk to the importance like in #272 is similarly a convenient way to indicate that we expect a requirement to not be applicable to all TREs.

We've discussed elsewhere how to handle different shapes of TRE. I could see a future where the spec filters based on characteristics of your TRE but I don't think we can solve that now. My intuition would be to keep it more general for now.

[craddm]

I think the asterisk is a reasonable way to do it for now.