brand new version

Author: gxywy

.gitignore

@@ -130,4 +130,5 @@ dmypy.json
 
 ## mine
 backup/
-rl_plotter-preview/
+rl_plotter-preview/
+rl_plotter-history/

README.md

@@ -2,7 +2,7 @@
 
 ![PyPI](https://img.shields.io/pypi/v/rl_plotter?style=flat-square) ![GitHub](https://img.shields.io/github/license/gxywy/rl-plotter?style=flat-square) ![GitHub last commit](https://img.shields.io/github/last-commit/gxywy/rl-plotter?style=flat-square)
 
- This is a simple tool which can plot learning curves easily for reinforcement learning.
+ This is a simple tool which can plot learning curves easily for reinforcement learning (RL).
 
 ## Installation
 
@@ -15,65 +15,66 @@ pip install rl_plotter
 from source
 
 ```
-python3 setup.py install
+python setup.py install
 ```
 
 ## Examples
 
-First, add a logger in your code (for example: DQN):
+First, add our logger (compatible with [OpenAI-baseline](https://github.com/openai/baselines)) in your code
+
+or just [OpenAI-baseline](https://github.com/openai/baselines) bench.Monitor (recommended)
 
 ```python
-from rl_plotter.logger import Logger
-
-def train(name):
-    dqn = DQN()
-    logger = Logger(name, env_name='PongNoFrameskip-v4', use_tensorboard=False)
-
-    while True:
-        s = env.reset()
-        while True:
-            total_step = logger.add_step()
-            a = dqn.select_action(s, EPSILON)
-            s_, r, done, info = env.step(a)
-
-            dqn.store_transition(s, a, r, s_)
-            episode_reward += r
-            
-            if dqn.replay_memory.memory_counter > REPLAY_MEMORY_SIZE:
-                loss = dqn.learn()
-                logger.add_loss(loss.cpu().item())
-            if done:
-                break
-            s = s_
-        logger.add_episode()
-        logger.add_reward(episode_reward, freq=10)
-    logger.finish()
+from baselines import bench
+env = bench.Monitor(env, log_dir)
 ```
 
 After the training or when you are training your agent, you can plot the learning curves in this way:
 
 ```
-python -m rl_plotter.plotter
+python -m rl_plotter.plotter --save --show
 ```
 for help use:
 ```
 python -m rl_plotter.plotter --help
 ```
 
-The learning curves looks like this:
+and you can find  parameters to custom the style of your curves.
+
+```
+optional arguments:
+-h, --help            show this help message and exit
+--fig_length          matplotlib figure length (default: 6)
+--fig_width           matplotlib figure width (default: 6)
+--style               matplotlib figure style (default: seaborn)
+--title               matplotlib figure title (default: None)
+--xlabel              matplotlib figure xlabel
+--xkey                x-axis key in csv file (default: l)
+--ykey                y-axis key in csv file (default: r)
+--smooth              smooth radius of y axis (default: 1)
+--ylabel              matplotlib figure ylabel
+--avg_group           average the curves in the same group and plot the mean
+--shaded_std          shaded region corresponding to standard deviation of the group
+--shaded_err          shaded region corresponding to error in mean estimate of the group
+--legend_outside      place the legend outside of the figure
+--time                enable this will set x_key to t, and activate parameters about time
+--time_unit           parameters about time, x axis time unit (default: h)
+--time_interval       parameters about time, x axis time interval (default: 1)
+--xformat             x-axis format
+--xlim                x-axis limitation (default: None)
+--log_dir             log dir (default: ./logs/)
+--filename            csv filename
+--show                show figure
+--save                save figure
+--dpi DPI             figure dpi (default: 400)
+```
+
+finally, the learning curves looks like this:
 <div align="center"><img width="400" height="400" src="https://github.com/gxywy/rl-plotter/blob/master/imgs/figure_1.png?raw=true"/></div>
-<div align="center"><img width="400" height="400" src="https://github.com/gxywy/rl-plotter/blob/master/imgs/figure_2.png?raw=true"/></div>
-<div align="center"><img width="400" height="400" src="https://github.com/gxywy/rl-plotter/blob/master/imgs/figure_3.png?raw=true"/></div>
-And you can custom the style of your curves by use parameter of `rl_plotter.plotter`or modifying`rl_plotter.plotter`
 
 ## Features
-- [x] reinforcement learning plot tools
-- [x] timestamp x axis features
-- [x] history experiment data plot tools
-- [x] x axis formatter features
-- [x] multiprocessing algorithm x.monitor  logger
-- [x] compatible with [OpenAI-baseline](https://github.com/openai/baselines) monitor data style
-- [ ] compatible with [OpenAI-baseline](https://github.com/openai/baselines) progress data style
-- [x] custom scalars logger (can be used to analyze any variable in training)
-- [ ] ~~basic data plot tools（including ML-Loss plot）~~
-- [ ] ~~dynamic plot tools~~
+- [x] custom logger, style, key, label, interval, and so on ...
+- [x] multi-experiment plotter
+- [x] x-axis formatter features
+- [x] x-axis formatter features
+- [x] compatible with [OpenAI-baseline](https://github.com/openai/baselines) monitor data style

imgs/figure_1.png

@@ -3,3 +3,4 @@ numpy==1.16.5
 statsmodels==0.10.1
 matplotlib==3.1.2
 tensorboardX==1.9
+glob

imgs/figure_2.png

@@ -6,19 +6,17 @@
 import csv
 import os
 import json
-import random
 import time
 import logging
-import matplotlib.pyplot as plt
 import numpy as np
 
 class Logger():
-    def __init__(self, exp_name, save=True, save_dir="./logs", env_name=None, use_tensorboard=False):
+    def __init__(self, exp_name, save=True, log_dir="./logs", env_name=None, use_tensorboard=False):
         if save:
-            self.save_dir = save_dir + "/" + exp_name + "/"
-            if not os.path.exists(self.save_dir):
-                os.makedirs(self.save_dir)
-            self.csv_file = open(self.save_dir + 'monitor.csv', 'w')
+            self.log_dir = log_dir + "/" + exp_name + "/"
+            if not os.path.exists(self.log_dir):
+                os.makedirs(self.log_dir)
+            self.csv_file = open(self.log_dir + 'monitor.csv', 'w')
             header={"t_start": time.time(), 'env_id' : env_name}
             header = '# {} \n'.format(json.dumps(header))
             self.csv_file.write(header)