1
$\begingroup$

If I will train one autoencoder with one vector only and second autoencoder with second vector only, does it mean if vectors were similar, that the hidden layer vectors of both autoencoders will be similar as well. Autoencoders structure is identical. The quantity of hidden layer neurons is smaller that input. So i want to reduce dimentionality. I understand that it will be overfitting. But for me it looks like you not learn features but just mimick your input function with the smaller dimensional vector while loosing information. So you receive kinda low dimentional approximation of learned vector. So if vector V1 was similar to V2 and I will use V1 to train Autoencoder A1 and V2 to train Autoencoder A2, does it mean that hidden layer vector of A1 and hidden Layer vector of A2 won't be that similar like V1 and V2 but still aproximately quite similar?

$\endgroup$
0
$\begingroup$

I'd say if you are training with a single vector, then no they wouldn't be the same.

It seems unintuitive at first but one has to realize it doesn't make sense to learn the features of a single vector, only the features of a set of vectors. The goal of an auto-encoder it to use its parameters to compress its dataset into a lower-dimension representation (A.K.A latent space) as efficiently as possible.

By efficiently, I mean that the most amount of useful information is encoded into the latent space. This means the vectors close in input space will also be close in the latent space. The auto-encoder achieves this by learning to utilize groups of commonly occurring pixels (e.g. features). For images, an A.E. might learn to describe a picture in terms of these-

http://colah.github.io/posts/2014-07-Conv-Nets-Modular/

Now instead of describing an image as a with all its pixels (i.e. the original image) we can instead describe it with lines of blobs of colors, effectively reducing the amount of information needed. Lines and blobs are learned instead of random pixels because lines and lines and blobs can describe any picture, but a random group of pixels can not.

As you can see, it is necessary when we have multiple vectors that we learn features that can approximate all the vectors in the dataset. And hence when we have only one vector we can use any combination of features because any combination of features will be able to approximate the original vector.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.