Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Codecov CLI can improve test collection when pytest is not in quite mode #343

Closed
rohan-at-sentry opened this issue Dec 19, 2023 · 1 comment

Comments

@rohan-at-sentry
Copy link
Contributor

Tests collections get malformed when pytest is not in quite mode.

@giovanni-guidini
Copy link
Contributor

When you run python -m pytest --collect-only -q the output looks something like below. We parse that, and actually use the lines as test labels.

core/tests/test_helpers.py::test_select_emotion_emoji
core/tests/test_helpers.py::test_select_emotion_default
core/tests/test_helpers.py::test_select_emotion_no_support_to_emoji
core/tests/test_helpers.py::test_select_emotion_fallback
[...]
fruits/tests/test_fruits_menu.py::TestFruitsMenu::test_menu_options

52 tests collected in 0.03s

If you don't use the -q flag though the output looks like below. Obviously harder to parse, so we don't parse that yet.

However there are many ways to config pytest, and it's possible that a user will have something to override the -q flag. So we need to able to parse the verbose version as well.

=============================================================== test session starts ================================================================
platform darwin -- Python 3.10.3, pytest-7.4.2, pluggy-1.3.0
rootdir: /Users/giovannimguidini/Projects/GitHub/components-demo
plugins: anyio-4.0.0, cov-4.1.0, mock-3.11.1, xdist-3.3.1
collected 52 items                                                                                                                                 

<Package tests>
  <Module test_helpers.py>
    <Function test_select_emotion_emoji>
    <Function test_select_emotion_default>
    <Function test_select_emotion_no_support_to_emoji>
    <Function test_select_emotion_fallback>
    <UnitTestCase TestGetFruitClass>
      <TestCaseFunction test_get_fruit_class_emoji_support_false>
      <TestCaseFunction test_get_fruit_class_emoji_support_no>
      <TestCaseFunction test_get_fruit_class_emoji_support_true>
      <TestCaseFunction test_get_fruit_class_emoji_support_yes>
    <UnitTestCase TestListAvailableMethods>
      <TestCaseFunction test_list_available_methods_with_instance>
      <TestCaseFunction test_list_available_methods_with_test_class>
 [...]
  <Module test_fruits_menu.py>
    <UnitTestCase TestFruitsMenu>
      <TestCaseFunction test_init_ascii>
      <TestCaseFunction test_init_emoji>
      <TestCaseFunction test_menu_choice_apple>
      <TestCaseFunction test_menu_options>

=========================================================== 52 tests collected in 0.03s ============================================================

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants