Tutor Adoption 4: Devstack Parity: Dev Workflows Option 1: Merge Quickdev

[kdmccormick]

Background

#144 describes three options for improving tutor dev workflows around locally-changed code and requirements. This is one option.

Tasks

Broadly speaking: Incorporate the behaviors of the quickdev plugin into Tutor as the default behaviors of tutor dev. Generalize the implementation so that plugins can hook up named volumes for services that they add (the quickdev plugin, as it is now, only works for edx-platform).

Specifically:

• Define named volumes for the LMS/CMS venv, egg-info, node_modules, and generated static assets. Declare them in the Dockerfile so that they’re auto-populated when LMS/CMS are first started.
• Add similar named volumes to the official Tutor plugins that add services: discovery, notes, mfe, etc.
• In the Tutor plugin tutorial, which adds a service to Tutor, add a step for defining named volumes.
• In order to prevent the contents of these named volumes from becoming stale over time, provide a mechanism for deleting them so that they can be freshly populated from the image. We could have an automatic way, a manual way, or both:
  • Automatic: Delete the named volumes regularly by triggering on a common event An aggressive version of this would be deleting the named volumes whenever the platform is stopped. A less-aggressive idea would be to clear them whenever launch is run, as well as whenever the related Docker image is pulled or built. We would need to add a few new hooks to implement this:
    • IMAGE_BUILT and IMAGE_PULLED actions, providing a way to to trigger the volume deletion; and
    • a COMPOSE_DEV_IMAGE_NAMED_VOLUMES filter in order to map images to their named volumes.
  • Manual: Create commands to clear out the named volumes on-demand. In quickdev, those commands are tutor quickdev [pip|npm|static]-restore, but they are not extensible. In Tutor, we could have tutor dev restore <image> <volume>. For example:
    • tutor dev restore openedx venv would delete the named volume holding the edx-platform virtual environment. This command could also use the new COMPOSE_DEV_IMAGE_NAMED_VOLUMES filter in order to map "<image> <volume>" to actual Docker volumes.
• Automatically mount suspected edx-platform packages to a standard location. In quickdev, the location is /openedx/mounted-packages, and the recognized prefixes are:
  • xblock-*, a fairly established pattern for XBlock repos,
  • platform-plugin-*, which 2U/edX started using recently and I like, and
  • platform-lib-*, which is not a pattern yet, but I wouldn’t mind encouraging it.
    The same thing could be done in plugins if we could figure out any reasonable naming conventions for their packages.
• Provide a command for installing all packages mounted in the standard location mentioned above In quickdev, that’s tutor quickdev pip-install-mounted -m ... -m .... In Tutor, this could be implemented using the upcoming CLI_DO_COMMANDS filter, giving us tutor dev do pip-install-mounted -m ... -m ... -m ....

Notes

Open question: Would the quickdev improvements translate easily to MFE development? If not, may need to create a follow-up story to specifically handle mounts & packages for MFE services.

[kdmccormick]

It was decided a while ago that we won't incorporate quickdev into Tutor proper.