My data contains a scalar feature $r$, I found this feature is important for training my deep model. My idea is supposing there is a 3-layer MLP $f(x), x \in \mathbb{R}^{n}$, where $n=1$. It outputs a vector with dimension $m$ where each value in $[0, 1]$.

For my data, it inputs $r$ and outputs an m-sized vector.

So here is my decomposition idea make sense?


Yes, you can do that by interchanging the position of decoder and encoder in an autoencoder.

In an autoencoder, you give a long vector as input - the encoder reduces it to a short length vector (compressed) - the decoder now takes this compressed vector as input and upsamples it to the size of the original vector. The autoencoder is trained by taking the Mean Square Error (MSE) of the output of decoder with respect to the input vector. This enforces the compressed vector representation to contain the information of the input vector.

Now coming to your case. You simply need to pass the single scalar value to a decoder that upsamples, say your 3 layer fully connected neural networks. Let this output be denotes as "latent representation". Now pass this "latent representation" to the encoder which uses this "latent representation" to output just a single scalar value. Use MSE objective to enforce the the above single scalar output to match the input scalar value. Once the training is done of the above reverse autoencoder, the "latent representation" will give you the required vector containing the information about the scalar value you wished to represent as a vector.

  • $\begingroup$ Hi, I only need the decoder one, that is decoding the scalar value to a low-dimension vector. Is it possible to use Autoencocer? $\endgroup$ – GoingMyWay Dec 22 '19 at 15:23
  • $\begingroup$ I don't think you can only use a decoder. You would also need an encoder which reconstructs the single scalar value from the low dimension vector. This is required to enforce dependence of the scalar value on the low dim vector much like autoencoders. $\endgroup$ – user1825567 Dec 23 '19 at 1:38
  • $\begingroup$ The model suggested above is like an autoencoder with the position of encoder and decoder reversed. In autoencoders, it is assumed that you are decreasing the dimension of the input hence the encoder preceds the decoder. But in your case, you are increasing the size so the decoder needs to precede the encoder. $\endgroup$ – user1825567 Dec 23 '19 at 1:43
  • $\begingroup$ Thank you, your answer is a good hint for me. $\endgroup$ – GoingMyWay Dec 23 '19 at 2:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.