Grading Structure

For each variant (consider <var_i>) the following is done:

1. Make an empty working directory (working/)
2. Load the files from tests/common/ into working/
3. Load the files from tests/<var_i>/ into working/
4. Load the solution into working/
   • in which _submission_file will be appended to the file detailed in meta.json
   • and if "submission_root" is defined in meta.json, all files are copied to working/<submission_root>
5. Run the GRADING_SCRIPT (in parse.py) and serialize the output into Var objects
   • GRADING_SCRIPT is defined in grader/parse.py and is by default $ cd working/ && bundle install --quiet && rspec --format json
6. Repeat steps 1-3
7. Load the student submission (which will be loaded into suites/submission upon launch of the image) into working/
   • the text input by the student will be in tests/submission/_submission_file
   • all other files will be in tests/submission/
8. Repeat step 5
9. Compare the Var object generated by the student's submission to that of the instructor's solution and report results to Prairielearn