1
$\begingroup$

From the paper

Human-level control through deep reinforcement learning, Mnih et al. Nature 2015

It says

Reinforcement learning is known to be unstable or even to diverge
when a nonlinear function approximator such as a neural network is
used to represent the action-value (also known as Q) function 20 .
This instability has several causes: the correlations present in the
sequence of observations

I am unsure how to understand this and cannot create any hypothetical examples where this can happen. What are some hypothetical scenarios, or real examples where the correlations present in the sequence disrupt the use of a 'deep learning' approximator is used?

$\endgroup$
1
$\begingroup$

One reason can be the high correlation between training data can cause bias the deep learner. In this way, if the learner gets a new data which is not correlated with the training data, it will have large error.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.