Is there a unittest adapter like asynctest for trio?

[lordi]

I have a whole test suite written with asynctest that I'd like to port to trio.

Is there a adapter class I could use? Simply decorating a unittest method with @trio_test did not work.

[njsmith]

There isn't currently any standard unittest/trio integration. It would be great if there were! As you correctly guessed, the idea of @trio_test was that it could be helpful for non-pytest users, but it's never really been fleshed out.

So you have a few options:

• The simplest thing to try would be to use pytest + pytest-trio to run your unittest-based testsuite. Pytest has explicit built-in support for this: https://docs.pytest.org/en/latest/unittest.html
  I'm not sure if pytest-trio and unittest will work together like this – if they don't then we can probably fix it so they do.

• You could write your own trio/unittest integration, similar to what asynctest does for asyncio. I imagine that a minimal viable version could be pretty simple, though it depends on what your test suite is doing.

• I don't know why @trio_test didn't work for you – maybe we could fix it?

If you say a bit more about how you're using asynctest and what your goals are (e.g. are you trying to avoid pytest for some reason?), then we might be able to say more. Also, what happened when you used @trio_test?

[lordi]

Wow, thanks for the elaborate response.

I love pytest and I'm currently running my unittest suite via pytest. So, pytest-trio would be my favourite way.

Here is an example test:

class PipesTests(unittest.TestCase):

    @trio.testing.trio_test
    async def test_once(self):
        ...

Would result in:

self = <unittest.case._Outcome object at 0x7fbdd87af1d0>, test_case = <tests.test_pipes.PipesTests testMethod=test_once>, isTest = True

    @contextlib.contextmanager
    def testPartExecutor(self, test_case, isTest=False):
        old_success = self.success
        self.success = True
        try:
>           yield

/usr/lib64/python3.7/unittest/case.py:59: 
-----
...
                with outcome.testPartExecutor(self, isTest=True):
>                   testMethod()
E                   TypeError: wrapper() takes 0 positional arguments but 1 was given

/usr/lib64/python3.7/unittest/case.py:615: TypeError

I think going for a simple wrapper class like asynctest would be my preferred way, too. I'll look into.

[njsmith]

TypeError: wrapper() takes 0 positional arguments but 1 was given

Hmm. I suspect I only wrote trio_test to work on top-level functions, and forgot to handle methods. Whoops.

I love pytest and I'm currently running my unittest suite via pytest. So, pytest-trio would be my favourite way.

...but if you're using pytest, then probably you're going to be better off getting pytest-trio working, instead of messing with trio_test. pytest-trio is solid and featureful and it's what we all use. The manual is here, and if you haven't seen it before it's worth at least skimming to get an idea of what it offers: https://pytest-trio.readthedocs.io/en/latest/

Unfortunately, it looks like right now pytest-trio can't automatically make async unittest testcases work. I tried enabling trio-mode and using this test file:

# test_demo_1.py
import unittest
import trio

class TestDemo(unittest.TestCase):
    async def test_demo(self):
        await trio.sleep(1)

...and it didn't work, I got:

$ pytest test_demo_1.py
============================= test session starts ==============================
platform linux -- Python 3.6.5, pytest-3.8.1, py-1.5.4, pluggy-0.7.1
rootdir: /tmp, inifile: pytest.ini
plugins: trio-0.5.1, repeat-0.5.0, instafail-0.4.0, cov-2.5.1, hypothesis-3.66.4
collected 1 item                                                               

test_demo_1.py .                                                        [100%]

=============================== warnings summary ===============================
/usr/lib/python3.6/unittest/case.py:605: RuntimeWarning: coroutine 'TestDemo.test_demo' was never awaited
  testMethod()

-- Docs: https://docs.pytest.org/en/latest/warnings.html
===================== 1 passed, 1 warnings in 0.02 seconds =====================

This means it didn't actually run the test. (Notice that it finished in "0.02 seconds", so it clearly didn't do sleep(1)...)

I suspect that pytest lets unittest.TestCase take control over running the test methods somehow, rather than how pytest normally runs tests, and pytest-trio doesn't have any special support for working with whatever unittest.TestCase is doing.

But, if I write it as a pytest test class, by simply deleting the base class:

# test_demo_2.py
import trio

class TestDemo:  # notice: no longer inherits from unittest.TestCase
    async def test_demo(self):
        await trio.sleep(1)

And now it works fine:

$ pytest test_demo_2.py
============================= test session starts ==============================
platform linux -- Python 3.6.5, pytest-3.8.1, py-1.5.4, pluggy-0.7.1
rootdir: /tmp, inifile: pytest.ini
plugins: trio-0.5.1, repeat-0.5.0, instafail-0.4.0, cov-2.5.1, hypothesis-3.66.4
collected 1 item                                                               

test_demo_2.py .                                                        [100%]

=========================== 1 passed in 1.02 seconds ===========================

So if your end goal is to run everything with pytest, then one option is to migrate away from unittest entirely – and depending on how heavily you're using unittest features, it might be pretty easy as well.

If that's not viable, then I guess the next best thing would be to figure out if we can make pytest-trio handle unittest-based test cases.

What do you think?

[smurfix]

FWIW, "move away from it" is what I habitually do with unittest.

The most annoying part is to replace all these .assertWhatever calls with assert … – but on the other hand pytest picks apart the asserted expression and shows a lot more information about the actual problem than unittest ever did, so that's a net time saver.

[njsmith]

There's no particular urgency to replacing the assertWhatever calls either; they work fine with pytest.

There's also this tool that claims to do it automatically, though I haven't tried it myself: https://github.com/pytest-dev/unittest2pytest

[smurfix]

There's no particular urgency to replacing the assertWhatever calls either; they work fine with pytest.

Except if you no longer inherit from unittest.TestCase …

[njsmith]

@smurfix haha wow you're right of course. I guess that shows how out-of-touch I am with unittest... I was mixing it up with the more modern, but still completely deprecated, style that nose tests used to use, where the assertWhatever were converted into top-level functions.

So I guess converting away from unittest is harder than I realized. Well, I opened an issue at pytest-trio to track this, though I'm not sure how urgently we should worry about it: python-trio/pytest-trio#74

[njsmith]

I don't think there's are any todo items for us here right now? @lordi Please re-open if you think of any (and let us know how it goes!)

[lordi]

Right, thanks for the pointers! I think porting the tests to non-unittest will be my go-to way.