Reinforcement Learning: An Introduction

R. S. Sutton and A. G. Barto

This repository contains a python implementation of the concepts described in the book Reinforcement Learning: An Introduction, by Sutton and Barto. For each chapter you will find a .py file that contains the main implementation, and a .ipynb used to quickly visualise figures on github.com.

The repository is still WIP. I will try to move linearly ahead with the book, you can check below for a roadmap of the immadiate actions.

Please, feel free to raise issues to ask questions or flag flaws and mistakes in the implementation.
Should you find this useful for you, I would be grateful if you'd star⭐ it :)

Available problems

Chapter 1: Introduction

• Tic tac toe

Chapter 2: Multi-armed Bandits

• Scheduled

Chapter 3: Finite Markov Decision Processes

• Scheduled

Chapter 4: Dynamic Programming (figures)

• GridWorld: Dynamic Programming Prediction with Iterative Policy Evaluation
• Jack's car rental: Dynamic Programming Control with Policy Iteration
• Gambler's Problem: Dynamic Programming Control with Value Iteration

Chapter 5: Monte Carlo Methods (figures)

• Blackjack: Monte Carlo Prediction with Policy Evaluation
• Blackjack: Single state-value estimation
• [WIP] Blackjack: Monte Carlo Control with Monte Carlo ES
• [WIP] On-Policy first-visit Monte Carlo Control
• [WIP] Off-Policy Monte Carlo Prediction via Importance Sampling
• [WIP] Off-Policy Monte Carlo Prediction with Policy Evaluation
• [WIP] Off-Policy Monte Carlo Control

References

[1] R. S. Sutton, A. G. Barto, et al. Reinforcement Learning: an Introduction. MIT press, Cambridge, 2018.
[2] Original Code, 2nd Edition. http://incompleteideas.net/book/code/code2nd.html