1
$\begingroup$

I'm studying this paper https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf, and I have a question about proposed algorithm(it appears on page 7 of the paper):

enter image description here

The problem I see is that author's function Q (and Q^) take a state as input and outputs values, corresponding to quality of all actions agent can take. However, in the algorithm the information is structured in the form (state, performed action, reward, new state), and this is used to compute target value yj. As I understand, yj must have [number of actions] dimensions, because that is what Q returns.

My question is how do I compute yj if I know reward only for one action(which was performed by agent)?

I found a tutorial https://keon.github.io/deep-q-learning/, where target values yj are initialized as Q(state) and yj[performed action] is initialized as in algorithm. I also implemented tutorial for snake game colab, agent learns slow, probably because of incorrect target values

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.