From the paper
Human-level control through deep reinforcement learning, Mnih et al. Nature 2015
Reinforcement learning is known to be unstable or even to diverge when a nonlinear function approximator such as a neural network is used to represent the action-value (also known as Q) function 20 . This instability has several causes: the correlations present in the sequence of observations
I am unsure how to understand this and cannot create any hypothetical examples where this can happen. What are some hypothetical scenarios, or real examples where the correlations present in the sequence disrupt the use of a 'deep learning' approximator is used?