CNN image to image translation: multiple image inputs to one image output

Question

I am interested in training a CNN to take in inputs where each input is a set of low-resolution images and each ground truth is a single high-resolution image. The ground truth high-resolution image was generated by averaging the information from a set of 80 low-level images. What I would like the CNN to do is to generate the same high-resolution image from a smaller set of low-level images, for example 5 low-level images.

I was looking into TensorFlow GAN and was thinking I could input each training image as a concatenated array of low-resolution images and each ground truth as a single high-resolution image. However, I am unsure whether this is the best method since the dimensions of the input and output will be different. Could someone advise what is the best approach using TensorFlow (or alternative deep learning frameworks if there is a better option)? Thanks!

Answer 1

GAN is a good choice also have a look at Autoencoder as well at the links below https://www.tensorflow.org/beta/tutorials/generative/cvae

and

https://towardsdatascience.com/deep-autoencoders-using-tensorflow-c68f075fd1a3

these two are very similar and will solve your problem statement for sure.