Create GitHub workflow to automatically run tests #64

-Added Dolos as submodule
-Modified how to compare matches:
 -Created a new Django command 'recompare' to compare matches without using the "Recompare all" or the "Recompare all exercises with unmatched submissions" button
  -In exercise settings "Recomparing all submissions" now recompares without needing the CLI command 'matchsubmissions'
   -In course page "Recompare all exercises with unmatched submissions" now recompares without needing the CLI command 'matchsubmissions'
-Added Python submissions for testing Radar
-Updated README.md for how to compare matches

.github/workflows/tests.yml

@@ -0,0 +1,52 @@
+name: tests
+on:
+  # trigger when pushed to any branch (for development)
+  push:
+  # trigger when a pull request is made to master
+  pull_request:
+    branches:
+      - master
+jobs:
+  playwright-tests:
+    timeout-minutes: 60
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v4
+      with:
+        submodules: recursive
+    - name: Set up Python
+      uses: actions/setup-python@v4
+      with:
+        python-version: '3.10'
+    - name: Start Dolos
+      working-directory: dolos
+      run: |
+        docker pull ghcr.io/dodona-edu/dolos-cli:latest
+        docker compose up --detach
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install -r requirements.txt
+    - name: Ensure browsers are installed
+      run: python -m playwright install --with-deps
+    - name: Set up Radar
+      run: |
+        python manage.py migrate
+        DJANGO_SUPERUSER_PASSWORD=Password \
+        python manage.py createsuperuser \
+        --no-input \
+        --username=Username \
+        [email protected]
+    - name: Upload submissions and compare
+      run: |
+        bash run_loadsubmissions.sh e2e_tests/testing_data/radar_test_python testcourse exercise1 1
+        python manage.py recompare testcourse/exercise1
+    - name: Run Radar
+      run: python manage.py runserver &
+    - name: Run tests
+      run: pytest --tracing=retain-on-failure
+    - uses: actions/upload-artifact@v4
+      if: ${{ !cancelled() }}
+      with:
+        name: playwright-traces
+        path: test_results/

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "dolos"]
+	path = dolos
+	url = https://github.com/dodona-edu/dolos

data/management/commands/recompare.py

@@ -0,0 +1,20 @@
+from django.core.management.base import BaseCommand
+
+from provider.filesystem import recompare
+from data.models import Course
+
+
+class Command(BaseCommand):
+    help = (
+        "Calculate similarity for all submissions in the exercise."
+    )
+
+    def add_arguments(self, parser):
+        parser.add_argument('course/exercise', type=str)
+
+    def handle(self, *args, **options):
+        (course_key, exercise_key) = options['course/exercise'].split("/", 1)
+        course = Course.objects.get(key=course_key)
+        exercise = course.get_exercise(exercise_key)
+
+        recompare(exercise, {})

e2e_tests/README.md

@@ -2,7 +2,7 @@
 
 This document provides a guide to create and run end-to-end tests for Radar using [Playwright](https://playwright.dev/python/).
 
-## Set up environment
+## Run Radar
 
 Make sure everything is set up following this [Radar development guide](https://github.com/apluslms/radar/blob/master/doc/DEVELOPMENT.md) before continuing.
 
@@ -17,22 +17,19 @@ Make sure everything is set up following this [Radar development guide](https://
 
 4. Load submissions: `./run_loadsubmissions.sh ${directory_with_submissions} {course} {exercise} 1`
 
-5. In a browser navigate to: `http://localhost:8000/{course}/{exercise}/settings/`
-    * Press the "Recompare all" button
+5. Match submissions: ` python manage.py recompare {course}/{exercise}`
 
-6. Match submissions: `python manage.py matchsubmissions {course}/{exercise}`
-
-## Set up Dolos
+## Run Dolos
 
 Dolos is required for `test_dolos` in `e2e_tests/test_dolos.py`.
 
-1. Clone the [Dolos](https://github.com/dodona-edu/dolos) repository
+Launch Docker daemon if Docker is not running: ` sudo dockerd`
 
-2. Launch Docker daemon: `dockerd`
+1. Change to Dolos directory: `cd dolos`
 
-3. In a new terminal at the root of the Dolos repository run: `docker compose up`
+2. Run Dolos: `docker compose up`
 
-Now Dolos should be running in a separate window and work with Radar.
+Now Dolos should be running in a separate window and works with Radar.
 
 ## Create tests
 

e2e_tests/testing_data/radar_test_python/A/ex2/1_file1_submission1

@@ -0,0 +1,10 @@
+import random
+
+def print_hello():
+    print('Hello, World!')
+
+def main():
+    print_hello()
+
+if __name__ == '__main__':
+    main()

e2e_tests/testing_data/radar_test_python/B/ex2/2_file1_submission2

@@ -0,0 +1,12 @@
+import random
+import math
+import datetime
+
+def print_hello():
+    print('Hello, World!')
+
+def main():
+    print_hello()
+
+if __name__ == '__main__':
+    main()

e2e_tests/testing_data/radar_test_python/C/ex2/3_file1_submission2

@@ -0,0 +1,11 @@
+import datetime
+import random
+
+def print_hello():
+    print('Hello, World!')
+
+def main():
+    print_hello()
+
+if __name__ == '__main__':
+    main()

e2e_tests/testing_data/radar_test_python/D/ex2/4_file1_submission1

@@ -0,0 +1,8 @@
+import random
+import math
+
+def add(a, b):
+    return a + b
+
+if __name__ == '__main__':
+    print(add(1, 2))

e2e_tests/testing_data/radar_test_python/E/ex2/5_file1_submission2

@@ -0,0 +1,9 @@
+import math
+import datetime
+import random
+
+def print_random_number():
+    print(f'Random number: {random.randint(1, 100)}')
+
+if __name__ == '__main__':
+    print_random_number()

e2e_tests/testing_data/radar_test_python/F/ex2/6_file1_submission1

@@ -0,0 +1,10 @@
+import random
+
+def print_hello():
+    print('Hello, World!')
+
+def main():
+    print_hello()
+
+if __name__ == '__main__':
+    main()

provider/filesystem.py

@@ -5,6 +5,8 @@
 from radar.config import tokenizer_config
 from provider.insert import submission_exists, insert_submission, prepare_submission
 from provider import tasks
+from matcher.tasks import match_exercise
+from radar.settings import DEBUG
 
 
 logger = logging.getLogger("radar.provider")
@@ -62,8 +64,10 @@ def recompare(exercise, config):
     exercise.course.save()
     exercise.clear_all_matches()
     exercise.touch_all_timestamps()
-    # Matching proceeds with CLI command
-    # matcher_tasks.match_exercise(exercise.id)
+    if DEBUG:
+        match_exercise(exercise.pk, False)
+    # Else matching proceeds with CLI command
+        # matcher_tasks.match_exercise(exercise.id)
 
 
 def recompare_all_unmatched(course):