I also asked this question on Stack Overflow. However, it has not yet been answered and I think this is a more suitable platform to place it.
I'm trying to implement a network set-up similar to this Google Deepmind paper. Their network set-up is as follows:
Mθ is a convolutional network, so I was wondering how they concatenate the input frames with the viewpoints? As far as I knew, the CNN takes into account the spatial information right? So does it even make sense to concatenate the frames with the viewpoints as an input for the CNN?
Thanks in advance!
Edit
I was thinking about it and they might have concatenated the viewpoint on the dense layers behind the convolution layers. Might that be the case?