Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create documentation on the curation process for new module approval #571

Open
mlocardpaulet opened this issue Feb 6, 2025 · 0 comments
Labels
to be discussed a decision still needs to be made

Comments

@mlocardpaulet
Copy link
Contributor

mlocardpaulet commented Feb 6, 2025

We need to have an independent expert opinion on new proposed modules.
This needs to be discussed.

The idea is to reach out to independent experts (proposed by the module proposal authors), and ask for their opinion on:

  • the data chosen for the benchmark
  • the metrics described an how they will be calculated
  • the input file format
  • the overall coherence
  • lack of redundancy with other modules

the comments can be done once or on the form of a dialogue with the developers. It should be public (in the github discussions).
experts can become collaborators and contribute to the module.
We need 2 forms for:

  • comments on the entire module proposal (concept, data, metrics, code)
  • OR comments only on the concept, data, metrics (for non coders).

This needs to be discussed among us, and we need to make documentation.

@mlocardpaulet mlocardpaulet added the to be discussed a decision still needs to be made label Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
to be discussed a decision still needs to be made
Projects
None yet
Development

No branches or pull requests

1 participant