Reinforcement learning algorithms

Question

I want to study and develop some application using reinforcement learning machine learning methods. I am already familiar with classification problem using supervised learning.

Can someone suggest me some material (links, youtube tutorials, pdf notes, ...) or some simple script in Rstudio (using maybe the iris dataset) to start studying from?

Thanks!

Answer 1

As your question was focused on reinforcement learning with RStudio I.e., in R language

BOOKS

• Hands on Reinforcement learning with R

You Tube

• Reinforcement Learn Techniques with R, packtpub tutorial series

  1. Reinforcement Learn Techniques with R : What Reinforcement Learning Can Do for You | packtpub.com

  2. Your First Reinforcement Learning Program

  3. Programming the Environment | packtpub.com

• Discover Algorithms for Reward-Based Learning in R | packtpub.com

  1. The Course Overview
  2. First model based program: Policy Evaluation and Iteration
  3. Programming model free environment using Monte Carlo & Q- learning
  4. Building Actions, Rewards, Punishments using Simulated Annealing Alt to Q-Learning

• Hands on Reinforcement learning with R | code in action (packt)

  1. Markov decision process in action
  2. Multi-Armed bandit models
  3. Dynamic programming for optimal policies
  4. Monte Carlo methods for prediction
  5. Temporal difference learning
  6. Reinforcement learning in Game applications
  7. MAB for financial engineering
  8. TD learning in healthcare
  9. Exploring deep reinforcement learning methods
  10. Deep Q learning using keras

PDF

• Reinforcement Learning in R
• Reinforcement Learning in R by Nicolas Pröllochs, Stefan Feuerriegel

Tutorial links

• HOW TO PERFORM REINFORCEMENT LEARNING WITH R

• Reinforcement Learning (Q-learning)

  1. An Introduction (Part 1)
  2. Implementation using R (Part 2)

COURSES

• Reinforcement learning with R:Algorithms-Agents-Environment-Udemy enter link description here
• Reinforcement learning specialisation Coursera

Lecture NOTES

• Reinforcement learning R slides
• Algorithms for Reinforcement Learning

OTHER GENERAL RESOURCES (not specific to R)

• The chapter by Bertsekas

• Mastering Reinforcement Learning with Python: Build next-generation, self-learning models using reinforcement learning techniques and best practices
• Reinforcement Learning Algorithms with Python: Learn, understand, and develop smart algorithms for addressing AI challenges
• Python Reinforcement Learning Projects: Eight hands-on projects exploring reinforcement learning algorithms using TensorFlow
• Reinforcement Learning: Industrial Applications of Intelligent Agents
• Handbook of Reinforcement Learning and Control: 325 (Studies in Systems, Decision, and Control)

• Algorithms for Reinforcement Learning: Csaba Szepesvari. Nice compendium of ready to be implemented algorithms.
• Reinforcement Learning and Dynamic Programming using Function Approximators. Busoniu, Lucian; Robert Babuska ; Bart De Schutter ; Damien Ernst (2010). This is a very practical book that explains some state-of-the-art algorithms (i.e., useful for real world problems) like fitted-Q-iteration and its variations.
• Reinforcement Learning: State-of-the-Art. Vol. 12 of Adaptation, Learning and Optimization. Wiering, M., van Otterlo, M. (Eds.), 2012. Springer, Berlin. In Sutton's words "This book is a valuable resource for students wanting to
go beyond the older textbooks and for researchers wanting to easily catch up with
recent developments".
• Optimal Adaptive Control and Differential Games by Reinforcement Learning Principles : Draguna Vrabie, Kyriakos G. Vamvoudakis , Frank L. Lewis. I am not familiar with this one, but I have seen it recommended.
• Markov Decision Processes in Artificial Intelligence, Sigaud O. & Buffet O. editors, ISTE Ld., Wiley and Sons Inc, 2010. I definitely suggest the books by Sutton and Barto as an excellent intro, the chapter by Bertsekas for getting a solid theoretical background and the book by Busoniu et al. for practical algorithms that can solve some non-toy problems. I also find useful the book by Szepesvari as a quick reference for understanding an comparing algorithms. There are also several good specialized monographs and surveys on the topic, some of these are:

• "From Bandits to Monte-Carlo Tree Search: The Optimistic Principle Applied to Optimization and Planning" by Remi Munos (New trends on Machine Learning). This monograph covers important nonconvex optimistic optimization methods that can be applied for policy search.

• "Reinforcement Learning in Robotics: A Survey" by J. Kober, J. A. Bagnell and J. Peters.

• "A Tutorial on Linear Function Approximators for Dynamic Programming and Reinforcement Learning" by A. Geramifard, T. J. Walsh, S. Tllex, G. Chowdhary, N. Roy and J. P. How (Foundations and Trends in Machine Learning).

• "A Survey on Policy Search for Robotic" by Newmann and Peters (Foundations and Trends in Machine Learning).

• markov decision process

• Algorithms for Reinforcement Learning (Synthesis Lectures on Artificial Intelligence and Machine Learning)

• Neuro-Dynamic Programming (Optimization and Neural Computation Series, 3)

Answer 2

The book "Reinforcement Learning" by Barto and Sutton is standard literature and was study material in at least two of my lectures. The presented algorithms are quite basic, giving you a proper foundation before you can delve into deep RL.

As soon as you have got solid understanding of the basics, here are good algorithms/papers about deep RL which I'd recommend reading in the order:

DDPG / CONTINUOUS CONTROL WITH DEEP REINFORCEMENT LEARNING by Lilicrap, Hunt et al.

PPO / Proximal Policy Optimization Algorithms by Schulman et al.

World Models / World Models by Ha and Schmidhuber

Dreamer / Dream to Control: Learning Behaviors by Latent Imagination by Hafner et al.

Note these papers are just a small part of many different approaches, but should give you a rough overview about what has been developed in the last years.