# Multiple Output Layers in Neural Networks in Deep Q Learning

I am trying to train a computer to play with LEGO bricks (simulated) using DQN. My input is an image with 4 color channels (RGB and depth) and the output of the neural network is the coordinates of the best position where the next brick should be put.

Instead of having $m_x * m_y * m_z$ (where $m_k$ is the size of positions along the k axis) different outputs, I want to have 3 output layers, each of which returns the x, y or z coordinates of the best position, so I have only $m_x + m_y + m_z$ outputs.

My idea is replacing the fully-connected output layer by three fully-connected output layers, but only have one reward value (labeled value) for three layers. Can I do this? Will the three layers returns different outputs if the reward is same?

You decide what the output layer represents, so it should be OK (and probably easier to implement) to have a single fully connected layer of size $m_x + m_y + m_z$ and to interpret the output in your own code. E.g. $n_x = N_{(x)}, n_y = N_{(m_x + y)}, n_z = N_{(m_x + m_y + z)}$, where $N_{(i)}$ is the output of the $i^{th}$ neuron. In the case of regression, this should have very similar behaviour to 3 separate fully connected layers for each dimension.
However, once you have either structure, it is not clear that you will be estimating reward directly any more. There is definitely no guarantee that the rewards will be the same between $n_x, n_y, n_z$, in fact they will not be in most cases. You will need to define a function $\hat{r}(x, y, z)$ that combined the outputs of the network and have your loss and error gradient calculated from that. The function could just be $\hat{r}(x, y, z) = n_x + n_y + n_z$ or maybe $\hat{r}(x, y, z) = n_x \times n_y \times n_z$
An alternative, if you don't want to build a network that has a large number of outputs, is to have a single reward estimate as the output, and have the action choice as an input. That has the advantage of leaving the details of the approximation to the neural network. It has the disadvantage that you would need to run the network up to $m_x * m_y * m_z$ times in order to decide the policy in each state.