From what I understand, a DQN agent has as many outputs as there are actions (for each state). If we consider a scalar state with 4 actions, that would mean that the DQN would have a 4 dimensional output.
However, when it comes to the target value for training the agent, it is usually described as a scalar value = reward + discount*best_future_Q.
How could a scalar value be used to train a Neural Network having a vector output?
For example see image in https://towardsdatascience.com/deep-q-learning-tutorial-mindqn-2a4c855abffc