2
$\begingroup$

I can't understand how is dimensionality reduction achieved in autoencoder since it learns to compress data from the input layer into a short code, and then uncompress that code into the original data I can' t see where is the reduction: the imput and the putput data have the same dimensionality?

$\endgroup$
15
$\begingroup$

Autoencoders are trained using both encoder and decoder section, but after training then only the encoder is used, and the decoder is trashed.

So, if you want to obtain the dimensionality reduction you have to set the layer between encoder and decoder of a dimension lower than the input's one. Then trash the decoder, and use that middle layer as output layer.

$\endgroup$
5
  • $\begingroup$ It's clear now. but what is the utility of decoder layer? $\endgroup$ – user Jun 13 '18 at 13:02
  • $\begingroup$ Also check here: towardsdatascience.com/deep-inside-autoencoders-7e41f319999f it is an unsupervised learning, there is no label here. So after encoding you decode by going back to the original input. Note that the output of the decoding won't be the same as the original input thus there will be a reconstruction error which is your loss function; and we always like to minimize the loss! Once it is converged, the output of encoding layer is used as compressed nonlinear representation of our data. $\endgroup$ – TwinPenguins Jun 13 '18 at 13:12
  • 5
    $\begingroup$ The decoder is useful only to train the encoder. This is the training process: the encoder propose a representation; the decoder try to reconstruct the data starting from that representation; the error between the original and the reconstructed data is back propagated; repeat until convergence (small error). Now you have an encoder able to build a meaningful representation of the data. With a lower dimension, in your case. $\endgroup$ – Vincenzo Lavorini Jun 13 '18 at 13:14
  • $\begingroup$ such a nice explanation @Vincenzo Lavorini, thanks $\endgroup$ – pcko1 Jun 13 '18 at 13:35
  • $\begingroup$ The decoder side isn't total junk, you can actually use them to (psuedo) fix corrupted files as seen here arxiv.org/pdf/1606.08921.pdf $\endgroup$ – CumminUp07 Jun 13 '18 at 21:05
0
$\begingroup$

For better understanding I have added a picture here. Auto encoder follows the strategy of neural network. Auto encoder is comprised with encoder and decoder. Half of it's encoder and remaining is decoder. According to the image there are multiple features x1, x2,x3 and encoder encoding it and providing z1 and z2 as output. So z1 and z2 is the encoded output. Reversely, if you decode again in the same way you will get the input. However, it may not be exact as the input. But, it's close to the input. Basically, depends on error rate. Auto Encoder

Auto encoder is used for multidimensional feature to be reduced as shown in the picture. The Decoder part is used to measure the auto encoder is preforming well or not. I mean measure the error rate of encoded features. Most of the time after training the model the decoder remains unused.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.