I am currently studying colorization of grayscale satellite imagery as part of my Master's internship. After looking for various machine learning techniques, I quickly decided to go for deep learning, as the colorization process can be completely automated and gives good results. I trained a wide variety of models and architectures (CNNs, Pix2Pix, DCGANs and WGANs with UNet or residual architectures, etc.) and settled with convolutional cGANs (condition: grayscal image ; output : colored image), but end up facing the same problem every time. Indeed, when training my network, the output is quite correct but always gives me grayish roofs.
I think it has something to do with the distribution of pixels values, as roofs are usually orange or black in my geographic area. Many articles state the fact that GANs are prone to mode collapse. In this case, there are two modes (orange and black) which probably results in a bad local equilibrium.
I tried many different techniques in order to improve training and/or avoid mode collapse :
- L1 loss regularization for the generator
- Label smoothing
- Noise taken from a normal distribution and concatenated to every layer in the generator/discriminator
- Dropout for some layers in the generator
- Gradient penalty
- Learning procedure inspired by DRAGANs and BEGANs
So far, gradient penalty has given me the best results, despite still having these grayish roofs. I know there are other ways of improving the training procedure (e.g. batch discrimination), but I do not have the pre-requisites in mathematics or computer science nor the time for implementing such techniques.
I was wondering if someone had some ideas for getting better results, or if someone also tried colorizing satellite imagery and ended up having such issues.
Thanks for reading this long message.
Have a nice day !
