When working with Q-Learning, what is the difference between having a Q_0(a) with all values zero, random or optimistic?


1 Answer 1


In the long-run, tabular Q-learning converges toward the optimal regardless of initialization.

However, the speed of convergence may be affected, similarly to an n-armed bandit setting : http://incompleteideas.net/book/first/ebook/node21.html

For more on initialization in Q learning, I recommend "Potential-based shaping and Q-value initialization are equivalent" by Eric Wiewiora.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.