Skip to content

Commit

Permalink
tests: constrain http pytest to tests/http directory
Browse files Browse the repository at this point in the history
Running the http pytest had to be done from tests directory or above,
because the repeat argument fixture was defined in tests/conftest.py.
However, the repeat argument is not needed because its functionality
can be provided by pytest-repeat as documented in the test's
README.md. So, removed the pytest_addoption function for the repeat
argument and the pytest_report_header function is moved to
tests/http/conftest.py.

TODO: Remove repeat argument from all tests. As a stopgap, a
one-element list is defined for it for now.

Closes curl#14611
  • Loading branch information
jan2000 authored and bagder committed Aug 22, 2024
1 parent aeb1a28 commit a415286
Show file tree
Hide file tree
Showing 6 changed files with 34 additions and 72 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/http3-linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -470,7 +470,7 @@ jobs:
env:
TFLAGS: "${{ matrix.build.tflags }}"

- run: pytest -v tests
- run: pytest -v tests/http
name: 'run pytest'
env:
TFLAGS: "${{ matrix.build.tflags }}"
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -487,10 +487,10 @@ jobs:
make V=1 VERBOSE=1 test-ci
- if: contains(matrix.build.install_steps, 'pytest')
# run for `tests` directory, so pytest does not pick up any other
# run for `tests/http` directory, so pytest does not pick up any other
# packages we might have built here
run:
pytest -v tests
pytest -v tests/http
name: 'run pytest'
env:
TFLAGS: "${{ matrix.build.tflags }}"
Expand Down
1 change: 0 additions & 1 deletion tests/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,6 @@ EXTRA_DIST = \
README.md \
appveyor.pm \
azure.pm \
conftest.py \
devtest.pl \
dictserver.py \
directories.pm \
Expand Down
64 changes: 0 additions & 64 deletions tests/conftest.py

This file was deleted.

8 changes: 4 additions & 4 deletions tests/http/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ This is an additional test suite using a combination of Apache httpd and nghttpx
The test cases and necessary files are in `tests/http`. You can invoke `pytest` from there or from the top level curl checkout and it will find all tests.

```
curl> pytest
curl> pytest test/http
platform darwin -- Python 3.9.15, pytest-6.2.0, py-1.10.0, pluggy-0.13.1
rootdir: /Users/sei/projects/curl
collected 5 items
Expand All @@ -24,7 +24,7 @@ tests/http/test_01_basic.py .....
Pytest takes arguments. `-v` increases its verbosity and can be used several times. `-k <expr>` can be used to run only matching test cases. The `expr` can be something resembling a python test or just a string that needs to match test cases in their names.

```
curl> pytest -vv -k test_01_02
curl/tests/http> pytest -vv -k test_01_02
```

runs all test cases that have `test_01_02` in their name. This does not have to be the start of the name.
Expand Down Expand Up @@ -54,10 +54,10 @@ Via curl's `configure` script you may specify:
Several test cases are parameterized, for example with the HTTP version to use. If you want to run a test with a particular protocol only, use a command line like:

```
curl> pytest -k "test_02_06 and h2"
curl/tests/http> pytest -k "test_02_06 and h2"
```

Several test cases can be repeated, they all have the `repeat` parameter (install `pytest-repeat` module). To make this work, you have to start `pytest` in the test directory itself (for some unknown reason). Like in:
Test cases can be repeated, with the `pytest-repeat` module (`pip install pytest-repeat`). Like in:

```
curl/tests/http> pytest -k "test_02_06 and h2" --count=100
Expand Down
27 changes: 27 additions & 0 deletions tests/http/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,33 @@

from testenv import Env, Nghttpx, Httpd, NghttpxQuic, NghttpxFwd

def pytest_report_header(config):
# Env inits its base properties only once, we can report them here
env = Env()
report = [
f'Testing curl {env.curl_version()}',
f' httpd: {env.httpd_version()}, http:{env.http_port} https:{env.https_port}',
f' httpd-proxy: {env.httpd_version()}, http:{env.proxy_port} https:{env.proxys_port}'
]
if env.have_h3():
report.extend([
f' nghttpx: {env.nghttpx_version()}, h3:{env.https_port}'
])
if env.has_caddy():
report.extend([
f' Caddy: {env.caddy_version()}, http:{env.caddy_http_port} https:{env.caddy_https_port}'
])
if env.has_vsftpd():
report.extend([
f' VsFTPD: {env.vsftpd_version()}, ftp:{env.ftp_port}, ftps:{env.ftps_port}'
])
return '\n'.join(report)

# TODO: remove this and repeat argument everywhere, pytest-repeat can be used to repeat tests
def pytest_generate_tests(metafunc):
if "repeat" in metafunc.fixturenames:
metafunc.parametrize('repeat', [0])

@pytest.fixture(scope="package")
def env(pytestconfig) -> Env:
env = Env(pytestconfig=pytestconfig)
Expand Down

0 comments on commit a415286

Please sign in to comment.