I am familiar with the concepts of Supervised and Unsupervised Learning but recently Reinforcement (Reinforced?) Learning also popped up before me a couple of times. Could anyone give a hint what is it, what is the main difference from the other two (or from the Unsupervised type, specifically), with some good example? Is it a promising alternative worth to explore or just some hyped niche curiosity?

Reinforcement Learning uses a simple logic of learning in which the network tries to learn from the feedback it obtains. This tries to optimise the overall reward in the long run instead of the current reward.

This is one of the best platform to read about it. It also contains some useful links.

As stated by the wiki, The basic reinforcement learning model consists of:

  • a set of environment states S;
  • a set of actions A;
  • rules of transitioning between states;
  • rules that determine the scalar immediate reward of a transition; and
  • rules that describe what the agent observes.

The rules are often stochastic. The observation typically involves the scalar immediate reward associated with the last transition. In many works, the agent is also assumed to observe the current environmental state, in which case we talk about full observability, whereas in the opposing case we talk about partial observability. Sometimes the set of actions available to the agent is restricted (e.g., you cannot spend more money than what you possess).

  • At first read it resembles me to neural networks, where one refines weights iteratively based on the cost. – Hendrik Oct 4 '16 at 11:23
  • First of all, you can have RL without a NN and the use of NNs within the 'environement-agent interaction loop' should not be confused with the actual RL method. I think the Wiki entry is a kind of misleading. I would briefly say that RL is an optimization method in which an agent tries to learn an optimal mapping of environmental states to actions (policy), with respect to an utility function. Also an RL model is not consisted of S,T,A. This is the description of a MDP in which the RL agent interacts and tries to solve. – Constantinos Jan 20 '17 at 12:42

Hima's answer does a good job summarizing the outline and purpose of reinforcement learning. If you are interested in taking a deeper look I'd recommend this currently free book.

It does a great job walking you from a basic reinforcement learning definition through various solutions to dealing with modern challenges.

Reinforcement learning is the intersection of machine learning, decisions & control, and behavioral psychology. The intersection can be approached from all the three sides. Let me give you a short description from every angle-

Machine Learning

From the ML perspective, RL is the paradigm of learning to control.

Think about how you learned to cycle or play a sport. These learning tasks are not supervised - no one tells you the correct move to make in a board position, or exactly the amount of angle to lean sideways to balance the cycle. They are also not completely unsupervised since some feedback is observed - whether you won or lost the game after a sequence of moves, how frequently do you fall from cycle.

Thus, RL is learning to make good decisions from partial evaluative feedback.

Control & Decision theory

In control theory (and AI planning), perfect knowledge about the world is assumed, and the objective is to find the best way to behave.

However, for many problems knowledge about the world is not perfect. Hence, exploring the world could increase our knowledge and eventually help us make better decisions.

RL is balancing the exploration-exploitation trade-off in sequential decision-making problems.

Behavioral Psychology

The simplified goal of behavioral psychology is to explain why, when, and how humans make decisions. We consider humans as rational agents, and hence psychology is also to some extent trying to explain rational behavior.

One can study the biological principles of how opinions are formed, which have close connections to temporal difference learning and eligibility traces.

RL is the paradigm to explain how humans form opinions and learn to make good decisions with experience.

This was a short description from every important perspective. For a detail description, kindly go through these --

Hope it helps!

Although the previous answers cover a lot to get you started in Reinforcement Learning (RL) field, I give you here an illustrative simple example to understand the concept and also what is the relationship between Supervised Learning (SL) and Unsupervised Learning (UL).

Imagine that you have a robot and you want to teach it to drive a car. Every let's say image of the road that the robot receives is going to be an input. One option that you have, in order to teach the robot, is that you can instruct it EVERY time that it receives the image of the road how much to steer the wheel. This is SL as you will have for every input state of the road a mapping to the proper angle of rotating the wheel. The main point here is that you know what is the optimal thing for your robot to do and you teach it by examples.

In a RL setting, you just let the robot try whatever it wants and you give it a reward/punishment regarding the action(s) it takes. The magnitude of the reward/punishment might be dependent on e.g. damage to the car, staying long time on the same lane etc. the reward/punishment might be given delayed and not at every single action that the robot takes.

In the first example (SL) the robot tries to minimize the error between your recommendation and its choices. In the second example the robot tries to maximize its reward by finding on its own what is the best to do. The SL approach at its best will lead you to a robot that "mimimcs" what you taught it. In the RL approach at its best, the robot will have a behavior that will be optimal in terms of driving the car and also might be better than yours. In other words it will create its own strategy.

To sum up,in SL you have a teacher that tells you at every single timestep exactly whats the correct response. In RL you try and find it on your own and the teacher gives you a reward/punishment. In UL you dont have any external feedback. So RL falls between SL and RL.

I simplified lots of the terms just to conceptualize the learning techniques with the example.

Hope it helps!

Reinforcement learning(RL) is the intersection of machine learning, decisions & control, and behavioral psychology. The intersection can be approached from all the three sides, and a detailed explanation is beyond the scope of a answer. SO, I'll try to give a short account of all the three perspectives. Reinforcement Learning (RL) field, I give you here an illustrative simple example to understand the concept and also what is the relationship between Supervised Learning (SL) and Unsupervised Learning (UL).

Imagine that you are teaching your child to play super mario and you have two option given the child the control stick and let him try to gain as many points as he can. or you play one level at a time and ask him to do as you did so as to reach the end of the level like you did, that is you teach him how to play one level and ask him to play that or a similar level of the game. This is SL as you will have for every input state of the road a mapping to the proper speed and points of jump and other attributes . The main point here is that you know what is the optimal thing and you try to teach your child to follow the similar steps. by doing so you teach him by examples.

In a RL setting, you just let the child try whatever it wants and you let the game give him a reward/punishment regarding the action(s) it takes.

In the SL example the child tries to minimize the error between your recommendation and its choices. In the RL example the child tries to maximize its reward by finding on its own what is the best to do. The SL approach at its best will lead you to a child to "mimics" what you taught it. In the RL approach at its best, the child will have a behavior that will be optimal in terms of playing Super Mario and also might be better than yours. In other words it will create its own strategy.

To sum up, in SL you have a teacher that tells you at every single time step exactly whats the correct response(think of math class where the teacher explains a identical problem to you on broad before giving you a problem to solve on you own). In RL you try and find it on your own and the teacher gives you a reward/punishment (think of a practical class where you are given a chemical and you have find its element/composition of the chemical and you follow you own intuition and do some trail and error to find the composition of the chemical. In UL you don't have any external feedback. So uL falls between SL and RL(think of bio class where teacher gives you example about some classes of animals and ask you to do some classification. say, dog, cat, lion are vertebrate but snail, crab, earthworm are invertebrate so without knowing the meaning of invertebrate and vertebrate tell me which of these are invertebrate and vertebrate: frog, snake and grasshopper.

I simplified a lots just to give you a hint on the learning techniques with the example.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.