I'm studying this paper https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf, and I have a question about proposed algorithm(it appears on page 7 of the paper):
The problem I see is that author's function Q (and Q^) take a state as input and outputs values, corresponding to quality of all actions agent can take. However, in the algorithm the information is structured in the form (state, performed action, reward, new state), and this is used to compute target value yj. As I understand, yj must have [number of actions] dimensions, because that is what Q returns.
My question is how do I compute yj if I know reward only for one action(which was performed by agent)?
I found a tutorial https://keon.github.io/deep-q-learning/, where target values yj are initialized as Q(state) and yj[performed action] is initialized as in algorithm. I also implemented tutorial for snake game colab, agent learns slow, probably because of incorrect target values