3
$\begingroup$

I am currently studying colorization of grayscale satellite imagery as part of my Master's internship. After looking for various machine learning techniques, I quickly decided to go for deep learning, as the colorization process can be completely automated and gives good results. I trained a wide variety of models and architectures (CNNs, Pix2Pix, DCGANs and WGANs with UNet or residual architectures, etc.) and settled with convolutional cGANs (condition: grayscal image ; output : colored image), but end up facing the same problem every time. Indeed, when training my network, the output is quite correct but always gives me grayish roofs.

I think it has something to do with the distribution of pixels values, as roofs are usually orange or black in my geographic area. Many articles state the fact that GANs are prone to mode collapse. In this case, there are two modes (orange and black) which probably results in a bad local equilibrium.

I tried many different techniques in order to improve training and/or avoid mode collapse :

  • L1 loss regularization for the generator
  • Label smoothing
  • Noise taken from a normal distribution and concatenated to every layer in the generator/discriminator
  • Dropout for some layers in the generator
  • Gradient penalty
  • Learning procedure inspired by DRAGANs and BEGANs

So far, gradient penalty has given me the best results, despite still having these grayish roofs. I know there are other ways of improving the training procedure (e.g. batch discrimination), but I do not have the pre-requisites in mathematics or computer science nor the time for implementing such techniques.

I was wondering if someone had some ideas for getting better results, or if someone also tried colorizing satellite imagery and ended up having such issues.

Thanks for reading this long message.

Have a nice day !

$\endgroup$
1
$\begingroup$

Colorfulness, specifically managing overly gray output, was touched on in the pix2pix paper, down in Section 4.2. It might be worthwhile to try out their formulation of a conditional GAN, as their results indicate that their objective function was useful in generating a more diverse set of colors.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Removing the L1 loss helped with having more colorful results. The model now generates a lot more artifacts, but more iterations will probably help with this problem. Thank you for your answer. Have a nice day ! $\endgroup$ – poqu67 May 24 '18 at 8:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.