2
$\begingroup$

I am abit confused by how autoencoders are able to reconstruct colors in images. According to me a CNN has feature detectors that convert the image into a sequence of feature or activation maps. These activation map corrospond to the degree of presense of a structural feature. Then how is the color information preserved in autoencoders if it is not a structural feature? To me the color information gets lost very quickly but most visuals of autoencoders show they preserve color very well.

$\endgroup$
2
$\begingroup$

Autoencoders ( AE ) are obviously unsupervised learning algorithms as they try to reconstruct the input or something similar to their input. To get a better understanding, we may use autoencoder to colourizing grayscale images. Given an input grayscale image, the AE will predict coloured images.

I have tried one such AE here.

Autoencoders learn to represent the given grayscale image into a latent ( compressed ) form. The encoder does this job. Now when the image has been brought down to its latent form, the decoder reconstructs the coloured image using that latent representation.

Due to this ability of autoencoders, they are used in image colorization, neural implanting and even generating images ( VAEs ). See this video.

Image colourization is not pixel-level operation. We need a smarter system which can literally observe the surroundings and assign a colour to the pixel. If we are training one such model on images of various landscapes ( sky and green croplands, for instance ) the most probable colour for green ( grass ) will be blue ( sky ).

Meaning, image colourization AEs extract spatial features which may be responsible for a colour change in the target image. A model trained on face images knows that there is a dark-coloured region above the eyes ( hair ). Many such features are learnt by even more complex models which give excellent results.

Autoencoders store the knowledge required to colourize an image by learning features from grayscale images. After enough they tend to know which region has to colourized with which colours.

$\endgroup$
1
  • $\begingroup$ I understand the case of black-white to colour images will require AE's to corelate structure to colour. I am speaking of full color inputs that are compressed to form the same image again. What kind of gabor filters are needed to preserve colour information? Gabor filters capture structural info. $\endgroup$ Oct 5 '19 at 12:40
1
$\begingroup$

Autoencoder is an unsupervised learning approach to compress the information/knowledge. Your knowledge is half-way there. One half - It finds the hidden attributes and prepares the compressed knowledge(Encoder). Second half it reconstructs the knowledge (Decoder). This way it will learn to reconstruct the knowledge. Hope it helps to give clarity.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.