0
$\begingroup$

Suppose I want to train an autoencoder on English words. Does it matter what kind of input data representation I use? E.g., if I use word2vec as my input or bag of words, will the quality of my autoencoder differ? If so, what do people typically use for autoencoder input?

$\endgroup$

1 Answer 1

0
$\begingroup$

Definitely, the input type of data can affect your autoencoder's performance.

On one side, the bag-of-words ignores context, but it is easy to train. On the other side, Word2Vec captures the semantic relationships, but again it's not that easy to train the model since it will need a lot of data and resources.

You may try One-hot encoding, since it's simple to use and usually used in autoencoders.

In general, there isn't necessarily a best choice here; what works best will depend on factors like your specific task requirements, available computational resources etc., so it's usually worth trying out several options if possible.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.