WeBWorK 2.20 Release Candidate #2721
Open
+35,760
−27,969
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This fixes some issues with #2511. This pull request is built on that one. That pull request is a nice start, but has some issues. Thanks to @ionparticle for starting this work. One of the most important things introduced in that pull request is the development identity provider via SimpleSAMLphp. That gave me a way to work with this. Although I liked the idea of using a Mojolicious plugin, that approach does not fit into the webwork2 course environment system. So, unfortunately, that had to be changed. So this rewrites the SAML2 authentication module to use the usual approach that all of the other authentication modules use. There is a `authen_saml2.conf.dist` file that should be copied to `authen_saml2.conf` to use SAML2 authentication. All settings for the authentication module are in that file. The primary limitation of the plugin approach was that it is not possible for different courses to use different identity providers. Or for one course to use SAML2 authentication and another only use the basic password authentication (without the need to add a URL parameter). This is something that is expected of the authentication modules, and is desirable for multi-institutional servers. Another problem with the implementation is that is does not work with `$session_management_via = "key"` set. The reason that doesn't work is because the implementation broke the rule of creating and accessing a session before authentication is completed. So this reimplementation uses the database via the "nonce" key hack much like LTI 1.1 authentication does. The previous implentation also used the `WEBWORK_ROOT_URL` environment variable in the code. That is not an environment variable that is defined for webwork2. It is only defined for the docker build, and can not be used in the code. The previous implementation also does not work with two factor authentication. If an identity provider does not provide two factor authentication, then webwork2's two factor authentication should be used. Of course, if the identity provider does provide two factor authentication, then the two factor authentication can be disabled in `localOverrides.conf`. An option to enable service provider initiated logout has also been added. It is off by default, but if enabled, when the user clicks the "Log Out" button, a request is sent to the identity provider to also end its session. The `README.md` file that was with the plugin code has been moved to the `docker-config/idp` directory, and is now purely for instructions on how to use the simpleSAMLphp development instance. The readme also includes instructions on how to set up a simpleSAMLphp development instance without docker. The documentation on how to configure and use the SAML2 authentication module is now all in the `authen_saml2.conf.dist` file. Note that the webwork2 service provider certificate files are no longer directly in the configuration file. Instead file locations are to be provided. It didn't really make sense to have the actual certificates in the configuration file, then write them to a file, which the Net::SAML2 module would read. Furthermore, it would be very easy to add the certificates incorrectly to the configuration file. With the YAML file approach if indentation were not correct, then the configuration file would fail to load.
Any route can now specify the methods that are allowed by adding a `methods` key to the route parameters. The value of the key should be a reference to an array containing the allowed methods. The ACS route is the only route that uses this at this point to restrict to the POST method only.
This option is for the case that the identity provider offers multi
factor authentication, and yet the $saml2{bypass_query} is also
allowed. In this case you would not want webwork2's two factor
authentication to be used when signing in via the identity provider.
However, two factor authentication should be used if the bypass query is
used. Setting $saml2{twoFAOnlyWithBypass} to 1 makes it so that
webwork2's two factor authentication is skipped for users signing in via
the identity provider, but still required for users signing in with a
username/password. If this is set to 0, then webwork2's two factor
authentication will always be required.
Really, the fix has been published in an update pg-codemirror-editor package (actually the bug was in the codemirror-lang-pg package). This just updates the version in the package.json file.
Add course admin tool to manage OTP secrets.
Updates to adding users to newly created courses.
Fix PGML Emphasis in the PG CodeMirror editor.
Hide the "Start New Test" button when acting as another user unless the user has permissions to do so. Fix some logic where the warning about creating new test version wouldn't show with the record_answers_when_acting_as_student permission, and instead the version would be created without warning. Translate the error messages that are created for an invalidSet. Remove the test for invalidProblem, which isn't used in tests. Update error messages to state 'test' instead of 'set'.
When acting as another user, disable the "Start New Test" button instead of hiding it if user doesn't have permission to start a test as a different user. Include a tooltip that states why the button is disabled.
Instead of telling user to click "Back Button", provide a "Cancel" button which returns them to the previous page. This also includes language updates as suggested during the PR review.
Tests, disable start button when acting as another user.
Fix invalid regex by moving hyphen to last character.
Improvements to SAML2 authentication.
This is a remnant from testing the CodeMirror 6 stuff locally, that seems to have stayed there even after uninstalling the local link. This was never supposed to have been committed.
Remove a local link in the `package-lock.json` file.
…user sets that were included in the scoring
add controls for when grades are sent to the LMS
This was broken in #2485. In that pull request line 16 of `templates/ContentGenerator/ProblemSet.html.ep` was changed from `<p class="mb-0"><%== $c->{invalidSet} %></p>` to `<p class="mb-0"><%= $c->{invalidSet} %></p>` which means that the invalid set message is now HTML escaped. That was necessary as almost all `invalidSet` messages include the set ID taken directly from the URL, and that is a cross-site scripting vulnerability. However, there is one message that does not use the set id from the URL, but does add HTML that needs to not be escaped. That is the message, `You must use your Learning Management System ([_1]) to access this set. Try logging in to the Learning Management System and visiting the set from there.` where the `[_1]` may be the LMS URL. That now displays as `You must use your Learning Management System (<a href="https://myschool.edu/lms/">the LMS</a>) to access this set. Try logging in to the Learning Management System and visiting the set from there.` `<%=` can certainly not be changed back to `<%==` because of the cross-site scripting vulnerability issue. However, there is another way to prevent HTML escaping. That is by using a `Mojo::Bytestream` object. So this message which is the only one that needs to not be HTML escaped (and is safe to do this with) is set in that way via the `b` method of a `Mojolicious::Controller`.
This commit just makes the changes in the scripts, but does not yet run perltidy with the new setup. That will be done in the next commit. Note that the `-vxl='q'` option has been removed. This option disables vertical alignment of qw quotes. Alignment of qw quotes was added in version 20220613 (the previous version), and this option to disable that was added to the webwork2 (and pg) perltidy configuration to avoid the alignment because it changes so many files. However, I don't think that alignment is bad, and I am tired of adding new options to prevent code changes. With this newer version another option would be needed to prevent the new signed number alignment (although that affects PG much more than webwork2). Note that the `bin/dev_scripts/run-perltidy.pl` script now insists on this specific version of perltidy instead of this version or newer. This is because every time a new version of perltidy comes out, it changes things. So developers really need to be using the same version or it will not be consistent with the github workflow result. Also, the version requirement in the check_modules.pl script has been removed. Only developers need a specific version of perltidy. Those running webwork2 can use other versions. This is only used for formatting problems in the PG problem editor, and using other versions may result in slightly different formatting there, but that is not so consequential that we need to require a specific version. Note that this change depends on the corresponding pg pull request, because in that pull request the `-vxl='q'` option has been removed. Otherwise version 20220613 or newer of perltidy would still be needed.
Fix the invalid set "You must use your LMS to access this set" message.
Switch to using Perl::Tidy version 20240903 (the latest).
Fix file uploads.
…eed-tweak Speed up assignments of achievements.
First get all of the necessary achievement data from the database, then process it and print it to the file. Comparing scoring all course achievements for all users in a course with 5000 users shows a considerable speed improvement. It takes more than three minutes for this with the develop branch, and less than 3 seconds with this pull request. Obviously the generated scoring files are identical.
The usual thing (by now with this series of pull request) is done. That is removing database access from loops and reducing to single queries as much as possible. In addition instead of using the Mojolicious tag helpers to render the "earned" checkbox and "counter" text field, direct html is used. I discovered that this renders much faster. With the usual 5000 user test the rendering of the template alone takes 5 seconds after changing a checkbox or counter value and saving, while with the direct html it takes something like 0.2 seconds. I believe that it has something to do with their code to set the values of the inputs that is causing the slow down. I am not sure on that, but I know that the direct html is much faster. That database changes are still the biggest part of the speed improvement here in any case. This page was rendering quite slowly even on initial load before, and that part was fast with the tag helpers. After making the database changes things sped up considerably, but then timing parts of the code when saving revealed the tag helper issue.
Speed up achievement scoring.
…ements Speed up the achievement users page.
…-restrictions Remove the filtering of large courses on the course admin "Manage OTP Secrets" page and fix security vulnerability.
Importing the `default_achievements.axp` file into a course with no achievements and 5000 users and assigning to all users took about 8 minutes with the WeBWorK-2.20 branch. With this pull request the time decreased to about 20 seconds.
…-speed-improvement Speed up the assignment of achievments when importing achievments.
…HTML renderer. This is an alternate for #2734, and implements the workaround suggested by @dpvc in mathjax/MathJax#3370 (comment).
The script now uses the settings in the course environment for external programs and to check the database driver (DBD::mysql or DBD::MariaDB). In order for that to work, first the script checks that all of the needed modules are found, load, and are at the sufficient version. That is except for the database driver module. If any are not found, then the script exits with a message stating that it is unable to continue without the required modules. If all modules are good, then the script checks the database driver module next. For this it "runtime" loads the course environment and its dependencies (as well as a runtime detection of the webwork2 root and lib directories). This is to address issue #2740. In addition the following modules are removed that are no longer needed: * Array::Utils * Email::Sender::Simple - still used but is a dependency of Email::Stuffer and doesn't need to be checked separately * HTML::Tagset * HTML::Template * IO::Socket::SSL * Net::LDAPS * Net::SMPT * Net::SSLeay * PadWalker * Path::Class * Safe - we use our local version (WWSafe.pm) and still check Opcode which it uses * Statistics::R::IO * Template `npm` is added to the list of required applications. Most of these things were noted in issue #1917.
Update the pg modules to include Plots modules.
Implement suggested workaround for erroneous extension loading with CHTML renderer.
update package lock for caniuse-lite
Fix two missed tabs in package-lock.json.
Rework check_modules.pl.
Remove old files (notes and problems) from the `/doc` directory
This was removed when the uploads were reworked, but I see now that it should not have been. Without this a warning about the empty string not being a valid upload occurs if there is not an upload.
Add back the "real" upload check.
move helper tool buttons to above problem list
Any time an achievement is created a warning is now displayed. This is due to an incorrect check for `$@` without a preceeding `eval`. This pull request adds the appropriate evals and corresponding checks for exceptions in `$@` which removes the warnings. Also, the delete handler was not correctly maintaining the `visibleAchievementIDs` list. As such, if you create an achievement, delete it, and then create an achievement with the same achievement ID again, two copies of the achievement are shown in the list (although only one actually exists in the datbase). So the delete handler now correctly maintains the `visibleAchievementIDs` list.
…sues Fix an inconsequential warning when a new achievement is created.
ability to cap the number of problems per page in a test
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It is time for this. We have talked about just using the develop branch, but I think it is nice to have a separate branch for the release candidate. It just keeps things separate in a nice way.
So as usual, re-target pull requests for this branch that you want to get into the release, and I will handle synchronizing those to develop as they are merged.