2
$\begingroup$

What is the difference between Reinforcement Learning (RL) and Supervised Learning?

Does RL hava more difficulty in finding a stable solution?

Does Q-learning have more difficulty in finding a stable solution?

Does getting stuck in a local minimum happen more in supervised learning?

Is this figure correct saying that Supervised Learning is part of RL?enter image description here

$\endgroup$
  • 1
    $\begingroup$ Could you link and give context to the diagram? Out of context, the supervised learning part seems quite wrong. Supervised learning models do not make decisions. They predict, which is different. $\endgroup$ – Neil Slater Sep 28 '18 at 7:30
2
$\begingroup$

What are difference between Reinforcement Learning (RL) and Supervised Learning?

The main difference is to do with how "correct" or optimal results are learned:

  • In Supervised Learning, the learning model is presented with an input and desired output. It learns by example.

  • In Reinforcement Learning, the learning agent is presented with an environment and must guess correct output. Whilst it receives feedback on how good its guess was, it is never told the correct output (and in addition the feedback may be delayed). It learns by exploration, or trial and error.

Could we say RL has more difficulty in [finding] a stable solution?

No, because the types of problems it solves are usually different, you cannot compare like with like.

However:

  • You could use RL as a framework to produce a predictive model for a dataset (using RL to solve a Supervised Learning problem). This would be inefficient, but it would work and there is no reason to expect it to be unstable.

  • In general, RL problems present additional challenges to supervised learning problems with the same degree of complexity in the relationship between input and output. They are "harder" in that sense, as more details need to be managed, there are more hyperparameters to tune.

  • RL can become unstable in ways that don't apply in supervised learning. For instance Q learning with neural network approximation tends to diverge, and needs special care (often experience replay is enough)

Could we say that [getting stuck] in [a] local minimum is seen more in supervised learning?

No. Internally, a RL agent will often use one kind of supervised learning algorithm to predict value functions. RL does not include any special features that can avoid or work around local minima. In addition, a RL agent can get stuck in ways that don't apply to supervised learning, for instance if an agent never discovers a high reward, it will create a policy that ignores entirely the possibility of getting to that reward.

Is this figure is correct saying that Supervised Learning is part of RL?

No. The figure is at best an over-simplified view of one of the ways you could describe relationships between the Supervised Learning, Contextual Bandits and Reinforcement Learning.

The figure is broadly correct in that you could use a Contextual Bandit solver as a framework to solve a Supervised Learning problem, and a RL solver as a framework to solve the other two. However, this is not necessary, and does not capture the relationships or differences between the three types of algorithm.

Otherwise the figure contains so much over-simplification and things that I would consider as errors (although this might just be missing context that explains the diagram), that I recommend you simply forget about trying to interpret it.

$\endgroup$
  • $\begingroup$ Thanks Neil, Could we say Q-learning has more difficulty in finding a stable solution? $\endgroup$ – user10296606 Sep 28 '18 at 9:22
  • 2
    $\begingroup$ @user10296606: In comparison to what? And what do you mean by "more difficulty"? That it fails completely without special care, or just takes longer? $\endgroup$ – Neil Slater Sep 28 '18 at 9:25
  • $\begingroup$ with Supervised learning $\endgroup$ – user10296606 Sep 28 '18 at 9:31
  • 1
    $\begingroup$ @user10296606: As explained in my answer, that is not really a comparison you can make, because you don't apply Supervised Learning and Q learning to the same problems. $\endgroup$ – Neil Slater Sep 28 '18 at 9:34
  • 1
    $\begingroup$ I think the OP still confuses SL and RL. Very simply put they solve different problems. RL solves optimization problems (need to learn to optimize an utility function by selecting appropriate actions) and SL solves prediction problems(learn a function that given inputs generates outcomes very close to your observations). Please note that I am not referring to the inner machinery of each method -- you can have predictive models in a RL algorithm and there is optimization for the model parameters -- but I point out the situations in which each method is employed. $\endgroup$ – Constantinos Sep 30 '18 at 0:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.