Rework problem data (or the persistence hash).

[drgrice1]

This makes it so that persistence hash is only saved to the database when answers are submitted. In the case that answers can be checked by the user (via the "Check Answers" button), then the persistence hash is saved JSON encoded in a hidden form field. That case does not need the security of saving to the database, and in that case it shouldn't be saved to the database because answers aren't being recorded and the data in that case should temporary. So a hidden form field is just right for this.

I objected to the previous implementation when #1940 was submitted, but allowed it at that time, but this is how the problem_data should have been implemented. Nothing should be written to the database for this when answers are not being recorded. If an instructor is acting as a student, the previous code was saving the persistent data to the student's problem even when the instructor just enters the problem. Of course it was also saving every time the instructor did anything with the problem including using the "Preview My Answers" button, the "Check Answers" button, the "Show Correct Answers" button, the "Show Problem Grader" button. Literally any form submission of the page. That just was not thought out.

The render_rpc (and html2xml) routes use the hidden form field approach to also support saving the problem data from the persistence hash. This means that problems that use the persistence hash can be tested in the PG problem editor.

Note that the PERSISTENCE_HASH_UPDATE flag is removed. That didn't improve efficiency at all. The processing that was done with that was too much. Even if this were saved when it was before it would have been more efficient to just save it.

The handling of the persistence hash is also reworked for PG in a corresponding pull request (see openwebwork/pg#1165). PG just sends the hash, and it can contain anything that can be JSON encoded. Webwork2 just JSON encodes it and stores it, but only when answers are submitted.

[drgrice1]

Here are some more problems for testing this. I am posting this in the webwork2 pull request because this emphasizes the changes made on this side. That is that the persistent data is no longer always saved, and the importance of that change.

Problem 1: persistent-data-deprecated.pg.txt

That problem will work with both the develop branches and with these pull requests. Add that problem to a set, and assign it to both a student user that you can test with as well as the instructor. Then when logged in as the instructor, act as the student, go to the problem, enter the correct answer, and click "Check Answers". The note below the problem will now say "You have 1 out of 5 correct so far."

If you do the above with the develop branches of webwork2 and pg, then sign out of the instructor account and sign in as the student (or sign in as the student in another browser). Go to the problem in the student's account. When you enter the problem the note below will say "You have 1 out of 5 correct so far." even though the student has never even entered the set yet.

Of course with the branches for these pull requests that won't happen. When acting as the student the notes below will still increase the number correct when you click "Check Answers". However, if you go to the student's account and look at the problem it will still say its initial message "You must get this problem correct 5 times for full credit."

Be careful when testing that you reset the problem when switching from develop to these pull requests or back (unassign and reassign).

Problem 2: persistent-data-updated.pg.txt

That problem is almost identical to the first. However, it will only work with these pull requests as it is updated to use the new and proper persistent_data method for utilizing the persistence hash that both gets and sets data, rather than the now deprecated store_persistent_data, update_persistent_data, and get_persistent_data methods.