Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moving to tox-docker #583

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Moving to tox-docker #583

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 2 additions & 10 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,18 +79,10 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install --upgrade tox tox-gh-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WisdomPill You may want to think about getting these out of your github actions and into something like a dev_dependencies (or pyproject.toml dev area). This way they're not as hidden from users.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I guess I will add some kind of dev script to setup the environment.

I still do not really understand how tox installs dependencies and where is it getting redislatest.

These are my first steps with tox

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Here's how I think about tox with docker:

  • Define a series of docker images that may or may not get used
  • Define tox environments that depend on individual dockers
  • Add pre_commands or dependencies if/as necessary
  • Run tests

For one project I use this pattern to parameterize a pytest option, so that we can control test execution and dockers being run. It may be a better example for your pattern.

python -m pip install --upgrade tox tox-gh-actions tox-docker

- name: Tox tests
run: |
REDIS_PRIMARY=$(tests/start_redis.sh)
REDIS_SENTINEL=$(tests/start_redis.sh --sentinel)
CONTAINERS="$REDIS_PRIMARY $REDIS_SENTINEL"
trap "docker stop $CONTAINERS && docker rm $CONTAINERS" EXIT
tests/wait_for_redis.sh $REDIS_PRIMARY 6379
tests/wait_for_redis.sh $REDIS_SENTINEL 26379

tox
run: tox
env:
DJANGO: ${{ matrix.django-version }}
REDIS: ${{ matrix.redis-version }}
Expand Down
3 changes: 3 additions & 0 deletions docker/base/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM redis:6.2.6-buster

CMD ["redis-server", "/redis.conf"]
3 changes: 3 additions & 0 deletions docker/base/Dockerfile.sentinel
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM redis:6.2.6-buster

CMD ["redis-sentinel", "/sentinel.conf"]
1 change: 1 addition & 0 deletions docker/base/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Dockers in this folder are built, and uploaded to the redislab dockerhub store.
2 changes: 2 additions & 0 deletions docker/master/redis.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
port 6379
save ""
6 changes: 6 additions & 0 deletions docker/sentinel/sentinel.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
port 26379

sentinel monitor default_service 127.0.0.1 6379 1
sentinel down-after-milliseconds default_service 10000
sentinel failover-timeout default_service 60000
sentinel parallel-syncs default_service 1
57 changes: 40 additions & 17 deletions setup.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ hiredis = redis[hiredis]>=3,!=4.0.0,!=4.0.1

[coverage:run]
omit =
# tests
tests/*,

[coverage:report]
Expand All @@ -66,17 +65,38 @@ multi_line_output = 3

[tox:tox]
minversion = 3.15.0
requires = tox-docker
envlist =
black
flake8
isort
mypy
# tests against released versions
# tests against released versions
py{36,37,38,39,310}-dj{22,31,32,40}-redislatest
# tests against unreleased versions
# tests against unreleased versions
py310-dj40-redismaster
py310-djmain-redis{latest,master}

[docker:master]
name = master
image = redisfab/redis-py:6.2.6-buster
ports =
6379:6379/tcp
healtcheck_cmd = python -c "import socket;print(True) if 0 == socket.socket(socket.AF_INET, socket.SOCK_STREAM).connect_ex(('127.0.0.1',6379)) else False"
volumes =
bind:rw:{toxinidir}/docker/master/redis.conf:/redis.conf

[docker:sentinel]
name = sentinel
image = redisfab/redis-py-sentinel:6.2.6-buster
links =
master:master
ports =
26379:26379/tcp
healtcheck_cmd = python -c "import socket;print(True) if 0 == socket.socket(socket.AF_INET, socket.SOCK_STREAM).connect_ex(('127.0.0.1',26379)) else False"
volumes =
bind:rw:{toxinidir}/docker/sentinel/sentinel.conf:/sentinel.conf

[gh-actions]
python =
3.6: py36
Expand All @@ -99,18 +119,20 @@ REDIS =
[testenv]
passenv = CI GITHUB*
commands =
{envpython} -m pytest --cov-report= --ds=settings.sqlite {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_json {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_lz4 {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_msgpack {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_sentinel {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_sharding {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_usock {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_zlib {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_zstd {posargs}
{envpython} -m coverage report
{envpython} -m coverage xml

{envpython} -m pytest --cov-report= --ds=settings.sqlite {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_json {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_lz4 {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_msgpack {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_sentinel {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_sharding {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_usock {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_zlib {posargs}
{envpython} -m pytest --cov-append --cov-report= --ds=settings.sqlite_zstd {posargs}
{envpython} -m coverage report
{envpython} -m coverage xml
docker =
master
sentinel
deps =
dj22: Django>=2.2,<2.3
dj31: Django>=3.1,<3.2
Expand All @@ -129,7 +151,8 @@ deps =

[testenv:{black,flake8,isort,mypy}]
basepython = python3
envdir={toxworkdir}/lint
envdir = {toxworkdir}/lint
docker =
commands =
black: black --target-version py36 {posargs:--check --diff} setup.py django_redis/ tests/
flake8: flake8 {posargs} setup.py django_redis/ tests/
Expand All @@ -142,7 +165,7 @@ deps =
isort >= 5.0.2
lxml
mypy
# typing dependencies
# typing dependencies
pytest
pytest-django
pytest-mock
Expand Down