Skip to content

Commit

Permalink
Add markdownlint to pre-commit (#1586)
Browse files Browse the repository at this point in the history
* Add markdownlint precommit hook

* Add markdown config
  • Loading branch information
irisliucy authored Jun 26, 2020
1 parent b7b9f78 commit 175ac4c
Show file tree
Hide file tree
Showing 8 changed files with 121 additions and 33 deletions.
7 changes: 6 additions & 1 deletion .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,13 @@ max_line_length = 80
indent_style = space
indent_size = 2

# shell scripts
# Shell scripts
[*.sh]
indent_style = space
indent_size = 2
max_line_length = 80

# Markdown
[*.md]
indent_style = space
max_line_length = 80
16 changes: 16 additions & 0 deletions .mdlrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Configuration file for markdown
# https://github.com/markdownlint/markdownlint/blob/master/docs/configuration.md
# Order of preference for markdownlint config file (first one wins):


all

# Creating style

exclude_rule 'MD013' # ignore line length = 80

exclude_rule 'MD004'# ignore unordered list style

exclude_rule 'MD012' # ignore multiple consecutive blank lines

exclude_rule 'MD034' # ignore bare URL used
10 changes: 10 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,16 @@ repos:
- id: pretty-format-json # Checks that all your JSON files have keys that are sorted and indented.
- id: trailing-whitespace # Trims trailing whitespace.

- repo: https://github.com/markdownlint/markdownlint
rev: v0.9.0
hooks:
- id: markdownlint # Check markdown files and flag style issues
name: Markdownlint
description: Run markdownlint on your Markdown files
entry: mdl
language: ruby
files: \.(md|mdown|markdown)$

# First pass: check format
- repo: https://github.com/pre-commit/mirrors-yapf
rev: v0.28.0
Expand Down
4 changes: 3 additions & 1 deletion docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,9 @@ RUN \
libbz2-dev \
libreadline-dev \
libssl-dev \
libsqlite3-dev && \
libsqlite3-dev \
# ruby
ruby-full && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

Expand Down
106 changes: 78 additions & 28 deletions docs/user/testing.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,28 @@
# Testing

In this section, we will talk about how to test different modules and algorithms in garage.
In this section, we will talk about how to test
different modules and algorithms in garage.

In garage, we use [pytest](https://docs.pytest.org/en/stable/getting-started.html) to carry out tests.
In garage, we use [pytest]<https://docs.pytest.org/en/stable/getting-started.html>
to carry out tests.

## Test Modules

All test files are stored under `tests/` in the garage respository.
All test files are stored under `tests/` in the
garage respository.

Test modules are organized in the same way as the garage main repository. Ideally, all main modules and files under `src/garage/` should be covered with test cases.
Test modules are organized in the same way as the
garage main repository. Ideally, all main modules
and files under `src/garage/` should be covered
with test cases.

Test modules of garage are structured in the following ways:

- `tests/` contains all tests and supporting code
- `tests/garage` contains unit tests
- `tests/integration_tests` contains integration tests
- modules in `tests/` and `tests/fixtures` contain helper codes and fixtures which make writing good tests easier.
- modules in `tests/` and `tests/fixtures` contain helper
codes and fixtures which make writing good tests easier.

```bash
tests
Expand All @@ -28,46 +35,73 @@ tests

## Prerequisites

To begin testing, we suggest using the same Python manager environment i.e. `virtualenv`, `conda` as we develop on garage repository so that all packages and dependencies required for testing are installed in the virutal environment.
To begin testing, we suggest using the same
Python manager environment i.e. `virtualenv`, `conda`
as we develop on garage repository so that all
packages and dependencies required for testing
are installed in the virutal environment.

Let's begin by activating the virtual environment:

```bash
# Conda
$ conda activate myenv
conda activate myenv

# Virtualenv
$ source myenv/bin/activate
source myenv/bin/activate
```

Next, we need to install pytest. Generally, pytest should have already been installed upon garage's `dev` installation. To install the garage environment for testing, you will need to install the `garage[all,dev]` for dependencies such as pytest, mujoco and so on.
Next, we need to install pytest. Generally, pytest
should have already been installed upon garage's
`dev` installation. To install the garage environment
for testing, you will need to install the
`garage[all,dev]` for dependencies such as pytest,
mujoco and so on.

Also, you may want to check out our installation guide first before diving into tests. It is also recommened to check out official [pytest](https://docs.pytest.org/en/stable/getting-started.html) documentation.
Also, you may want to check out our installation guide
first before diving into tests. It is also recommened
to check out official
[pytest](https://docs.pytest.org/en/stable/getting-started.html) documentation.

```bash
$ cd path/to/garage/
$ pip install -e '.[all,dev]'
cd path/to/garage/
pip install -e '.[all,dev]'
```

To get started, run pytest as follows:

```bash
$ pytest
pytest
```

## Basic Pytest Usage

Congrats! Now you're ready for testing garage!

We will start with a simple test. To be consistent with pytest requirement and garage modules, we name our test `.py` file with pre-fix `test_`.
We will start with a simple test. To be consistent
with pytest requirement and garage modules,
we name our test `.py` file with pre-fix `test_`.

Let's write a simple test case for the vpg algorithm `src/garage/tf/algos/vpg.py`.
Let's write a simple test case for the vpg
algorithm `src/garage/tf/algos/vpg.py`.

We begin by creating a file called `test_vpg.py` and put it under the tree `tests/garage/tf/algos/`.

We want to test VPG in the cart pole environment. Hence, we create a unitest named `test_vpg_cartpole`. Inside the function, we define the environemnt, the policy and the baselien and feed them to the VPG algorithm and run the experiment. Finally, we make sure the return value is identical to our expectation by using `assert`.

In new xunit-style tests, multiple tests are modeled into a class for modular and scalable structure. There is no need to subclass or anything, but make sure the prefix of the class starts with `Test`, otherwise the class will be skipped. Note that it's not encouraged to use this style, especially when a test doesn't require `setup_method` and `teardown_method`.
We want to test VPG in the cart pole environment.
Hence, we create a unitest named `test_vpg_cartpole`.
Inside the function, we define the environemnt,
the policy and the baselien and feed them to the
VPG algorithm and run the experiment. Finally,
we make sure the return value is identical to our
expectation by using `assert`.

In new xunit-style tests, multiple tests are modeled
into a class for modular and scalable structure.
There is no need to subclass or anything,
but make sure the prefix of the class starts
with `Test`, otherwise the class will be skipped.
Note that it's not encouraged to use this style,
especially when a test doesn't require `setup_method` and `teardown_method`.

```python
# test_vpg.py
Expand Down Expand Up @@ -103,49 +137,61 @@ class TestVPG(...):
In general, we can start running tests simply by:

```bash
$ pytest
pytest
```

However, in most of the cases, we simply don't have the time to test everything, plus Travis CI will take care of the majority of tests upon the deployment workflow. We can use the following ways to carry out specific tests to make life easier.
However, in most of the cases, we simply don't have
the time to test everything, plus Travis CI will
take care of the majority of tests upon the
deployment workflow. We can use the following
ways to carry out specific tests to make life easier.

### Specifying Tests / Selecting Tests

Pytest supports several ways to run and select tests.

#### Run tests in a directory

Run a test on particular module(s) by specifying a directory path.
Run a test on particular module(s) by specifying
a directory path.

```bash
$ pytest tests/garage/tf/algo/
pytest tests/garage/tf/algo/
```

#### Run tests in a module

Run a test on a particular module by specifying a file path.

```bash
$ pytest tests/garage/tf/algo/test_ppo.py
pytest tests/garage/tf/algo/test_ppo.py
```

#### Run tests by keyword expressions

Run tests by keyword expressions. This is useful for running particular test function(s).

```bash
$ pytest -k test_ppo_pendulum_continuous_baseline
pytest -k test_ppo_pendulum_continuous_baseline

```

## Useful Pytest Methods

Below are the pytest methods and functions that we found helpful for testing garage.
Below are the pytest methods and functions that
we found helpful for testing garage.

### Setup and teardown methods

`setup_method` is called before every tests to set up the test environment. It setups any state tied to the execution of the given method in a class. `setup_method` is invoked for every test method of a class.
`setup_method` is called before every tests to
set up the test environment. It setups any state
tied to the execution of the given method in
a class. `setup_method` is invoked for every
test method of a class.

`teardown_method` is called after every tests to teardown any state that was previously setup with a `setup_method`.
`teardown_method` is called after every tests to
teardown any state that was previously setup
with a `setup_method`.

For details on `setup_method` and `teardown_method`, check [this](https://docs.pytest.org/en/2.8.7/xunit_setup.html).

Expand All @@ -164,7 +210,11 @@ class TestSampleClass:

### Parametrized test functions

Parametrized test function is a delightful solution to save us from tedious testing in same scenarios with different parameters. We can simply specify the name of the arguments that will be pass to the test function and a list of arguments corresponding to the names.
Parametrized test function is a delightful solution to
save us from tedious testing in same scenarios with
different parameters. We can simply specify the name
of the arguments that will be pass to the test function
and a list of arguments corresponding to the names.

```python
import pytest
Expand Down
3 changes: 2 additions & 1 deletion scripts/setup_colab.sh
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,8 @@ sudo apt install -y \
swig \
libsdl2-dev \
libopenmpi-dev \
openmpi-bin
openmpi-bin \
ruby-full

# Build GLFW because the Ubuntu 16.04 version is too old
# See https://github.com/glfw/glfw/issues/1004
Expand Down
4 changes: 3 additions & 1 deletion scripts/setup_linux.sh
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ BASH_RC="${HOME}/.bashrc"
# Required for mujoco_py: libglew-dev, patchelf, libosmesa6-dev
# Required for OpenAI gym: libpq-dev, ffmpeg, libjpeg-dev, swig, libsdl2-dev
# Required for OpenAI baselines: libopenmpi-dev, openmpi-bin
# Required ruby for markdown lint tool
echo "Installing garage dependencies"
echo "You will probably be asked for your sudo password"
sudo apt -y -q update
Expand All @@ -140,7 +141,8 @@ sudo apt install -y \
swig \
libsdl2-dev \
libopenmpi-dev \
openmpi-bin
openmpi-bin \
ruby-full

# Build GLFW because the Ubuntu 16.04 version is too old
# See https://github.com/glfw/glfw/issues/1004
Expand Down
4 changes: 3 additions & 1 deletion scripts/setup_macos.sh
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,7 @@ hash brew 2>/dev/null || {
# For building glfw: cmake
# Required for OpenAI gym: cmake boost boost-python ffmpeg sdl2 swig wget
# Required for OpenAI baselines: cmake openmpi
# Ruby required for markdownlint tool
brew update
set +e
brew install \
Expand All @@ -192,7 +193,8 @@ brew install \
ffmpeg \
sdl2 \
swig \
openmpi
openmpi \
ruby
set -e

# Leave a note in ~/.bash_profile for the added environment variables
Expand Down

0 comments on commit 175ac4c

Please sign in to comment.