Timeouts for non-terminating tests

[debbieyuster]

Does Otter include timeouts to ensure the grading process doesn't get stuck indefinitely in a particular test? For example, suppose a test calls a student's function, and that function contains an infinite loop. Gradescope will eventually timeout the autograder, but I'm wondering whether Otter Grader will give up on the test that is hanging (and declare it as Failed) and move on to the other tests. If so, where exactly in the autograder files is this configured/specified? I'm asking because I'd like to implement this behavior in a non-Otter autograder, and am hoping to replicate whatever Otter does to ensure the autograder doesn't get stuck on a particular test case.

[chrispyles]

Otter doesn't support timeouts on a test-by-test basis; it just relies on Gradescope's global timeout, or there is a CLI flag for local grading

[lancemathias]

In our course, we used the signal module built into Python and the alarm function to trigger an interrupt and give a descriptive error message after a specified amount of time.
However, this only works on Linux/Unix operating systems and won't work on Windows.

[debbieyuster]

Thanks! This is very helpful.