Skip to content

Reinforcement Learning Tutorials & other bedtime stories in PyTorch

License

Notifications You must be signed in to change notification settings

moabitcoin/cherry-pytorch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

7a05b48 Β· Feb 12, 2020
Jan 6, 2020
Feb 12, 2020
Feb 12, 2020
Jan 6, 2020
Jan 9, 2020
Jan 13, 2020
Jan 13, 2020
Jan 13, 2020
Jan 13, 2020
Jan 28, 2020
Jan 13, 2020
Jan 29, 2020
Feb 12, 2020
Feb 12, 2020
Feb 12, 2020
Feb 12, 2020
Jan 6, 2020

Repository files navigation

πŸ’ Cherry in PyTorch

Reinforcement Learning & other bedtime stories in PyTorch. This repository is pytorch implementation of the Intro RL course & Advanced RL course. The name cherry is derived from Yann Lecun's statement regarding RL being the cherry on the ML cake. For training/playing the agent within conda env, please install the external dependencies. Not to be confused with a package of similar name.

Table of Contents

πŸ’» Installation

Tested with ubuntu:18.04 only

🐍 Conda

# After installed external dependencies
conda create -n cherry python=3.6
conda activate cherry
pip install -r requirements.txt
pip install -e .

🐳 Docker

# CPU
docker pull moabitcoin/cherry-pytorch-cpu
docker run moabitcoin/cherry-pytorch-cpu cherry --help
# GPU
docker pull moabitcoin/cherry-pytorch-gpu
docker run moabitcoin/cherry-pytorch-gpu cherry --help

🐳 Docker (build from scratch)

# CPU
make install dockerfile=Dockerfile.cpu dockerimage=moabitcoin/cherry-pytorch-cpu
make run dockerimage=moabitcoin/cherry-pytorch-cpu
# GPU
make install dockerfile=Dockerfile.gpu dockerimage=moabitcoin/cherry-pytorch-gpu
make gpu dockerimage=moabitcoin/cherry-pytorch-gpu

πŸŽ‰ Usage

# Inside the virtual environment / Docker
cherry --help
cherry train --help
usage: cherry train [-h] -c CONFIG_FILE [-d {gpu,cpu}] [-l {info,debug}]

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG_FILE, --config_file CONFIG_FILE
                        Path to Config file (default: None)
  -d {gpu,cpu}          Device to run the train/test (default: gpu)
  -l {info,debug}, --log {info,debug}
                        Set verbosity for the logger (default: info)

πŸ‘€ Example (Cartpole)

Train

cherry train -c configs/control.yaml -d cpu

Play

# <model_dest> in configs/control.yaml
cherry play -c <model_dest>/control-<commit-gitsha>.yaml -d cpu -m <model_dest>/agent-final-<commit-gitsha>.pth

Visualise

# <state_dest> in configs/control.yaml
vlc <state_dest>

Architectures

We support 4 feedforward architectures within cherry. We plan to expand the list of architectures to include Recurrent/Transformer/Memory architectures. If your personal model flavour is missing, please open an issue with links to architecture details.

Agents

We support 4 Agents within cherry. We plan to expand the list of agents to include TRPO/PPO. Please feel free to make an agent request by opening an issue with useful links to publication(s)/existing implementation.

Environments

We support 4 environments within cherry. This list is planned to be expanded to include Robotics and other 3rd party environments.

πŸ‘¬ Dependencies

:godmode: ViZDoom

Install dependencies as outlined here

sudo apt-get install libcanberra-gtk-module
pip install vizdoom
# If on a headless machine
export DISPLAY=:0

πŸ‘Ύ Arcade Learning Environment

https://github.com/openai/gym/blob/master/docs/environments.md#atari

πŸ“ˆ OpenAI RL baselines

Needed for Deepmind styled pre-processing wrappers. Needs TF>=1.14 πŸ€¦β€β™€οΈ

pip install tensorflow_gpu==1.14
https://github.com/openai/baselines/tree/master#installation

πŸ“Ί Remote Render

https://github.com/openai/gym/tree/0.15.4#rendering-on-a-server
xvfb-run -s "-screen 0 1400x900x24" bash