# Dimensionality of the target for DQN agent training

From what I understand, a DQN agent has as many outputs as there are actions (for each state). If we consider a scalar state with 4 actions, that would mean that the DQN would have a 4 dimensional output.

However, when it comes to the target value for training the agent, it is usually described as a scalar value = reward + discount*best_future_Q.

How could a scalar value be used to train a Neural Network having a vector output?

For example see image in https://towardsdatascience.com/deep-q-learning-tutorial-mindqn-2a4c855abffc

• can you provide the reference of the statement: "..a DQN agent has as many outputs as there are actions (for each state)" ? – Nikos M. Mar 28 at 17:39
• See the image on analyticsvidhya.com/blog/2019/04/… for example. The DQN outputs match the number of actions. – Dhoop Mar 28 at 18:18
• I doubt the image reflects this, as it has a single node as output. My guess is that it simply shows the possible outputs (of that single node) – Nikos M. Mar 28 at 18:20
• Are you sure you are looking at the right image? I have no way to paste the image here. Also look at many other blogs such as towardsdatascience.com/… If you dont agree with this statement, then can you state how many outputs you think a DQN agent has? – Dhoop Mar 28 at 18:23
• I have edited your post to add the image, unfortunately it has to await approval – Nikos M. Mar 28 at 18:25

I am of the opinion that this architecture is only one among others that can solve the same problem (for example one may have only 2 outputs one for the chosen action and one for the $$Q$$ value of that action, but I will not elaborate further on this).

What this architecture does is to output the whole function $$Q(a)$$, that is the $$Q$$ value as a function of action $$a$$. So each output node represents the $$Q$$ value for a certain action $$a$$ corresponding to that node (so node 1 corresponds to the $$Q$$ value for action $$a_1$$, node 2 to $$Q$$ value for action $$a_2$$ and so on..)

BUT this is NOT a vector output, in the usual sense of the term. This is a functional output that represents the whole function $$Q(a)$$ for each action $$a$$.

As far as the learning/decision rule is concerned things are as usual. Hope this is clear.