Generative adversarial networks for multiple distribution noise removal

Question

I am working on a project where I need to denoise images, and my dataset is composed of a big chunk of pairs <natural image, same image with synthetically added noise>. The fact is, I have multiple sources of noise (with labels), e.g. Gaussian noise, salt&pepper, distortion, saturation to name a few. Different noises types are on the same original images, meaning that for each undistorted image, I have one pair for each noise type, which I believe is relevant for training.

Thanks to the recent success of GANs for image translation tasks, I was looking into recent architectures and how I can adapt them for my task. The question is: do you think that is possible for a GAN to learn a many-to-one mapping between distributions, i.e. different noise distributions (the many) and the undistorted image distribution (the one) or do I need to train multiple networks for different kinds of noise?

Given that I have paired images I was looking into the pix2pix architecture. What about the recent CycleGAN? Learning the backward mapping function looks a lot tougher if there are multiple distributions, which I guess is why when they release the pre-trained models they indeed have multiple models for multiple tasks.

Answer 1

The task is denoising, so I think you can go with many-to-one mapping between distribution, and then I would probably try changes in generatr architecture if it won't learn properly. With multiple models, you'd have to classify the type of noise to select correct trained model for particular noise distribution.

And about the CycleGAN...

CycleGAN is for unpaired images. It tries to learn generally more complex problem, because it does not require paired images.

Since you have paired images, you should utilize that fact and go for pix2pix which needs paired images. Also, CycleGAN mas 2 generators and 2 discriminators, so you have bigger model and thus you need more computational power.