Comparison of results

[Yuri05]

If 7.2.2-installation validator is started on a system, where 7.2.1-validator run was performed (using the same output folder), validation fails, because in 7.2.2 some obsolete simulations were removed.

Obviously Installation Validator uses users Output folder as "master", which is wrong.
It should use the installed output folder as master

[msevestre]

This is not the issue. The tool always compare files in both folders. Since calculation were already performed , some previous files are in the folder already

The tool should empty the output folder before running the comparison. Don't you think that this is the problem?

[msevestre]

@Yuri05 There is no master. Missing simulations are shown from both folders. This could be changed but the concept of master folder is not there

[msevestre]

Which also shows that one should not remove stuff between two update release

[Yuri05]

Missing simulations are shown from both folders. This could be changed but the concept of master folder is not there

If missing sims are shown for both folders it would be sufficient to delete all json files from the output folder.

[UlrichSi]

Cleaning up behind program execution sounds like a good idea. Our local (SSD) drives fill up pretty quickly anyhow.

[Yuri05]

Cleaning up behind program execution sounds like a good idea. Our local (SSD) drives fill up pretty quickly anyhow.

We might need the outputs for investigation in case of failures

[UlrichSi]

True - maybe instead cleaning up the output of previous runs when a new validation run is started?

[Yuri05]

True - maybe instead cleaning up the output of previous runs when a new validation run is started?

In my opinion this should not happen automatically. We could introduce a button "Delete results from previous validations" or something like that. If the user never changed a default output folder, those can be located and cleaned up. (But I am not fan of this)

[msevestre]

So what should we do here? The error only happens bercause we did not respect semantic versioning and removed stuff in a minor release. In my opinion there is nothing to change