0
$\begingroup$

if Word2Vec is nothing but a transformation of one-hot into a dense vector, why can't I just feed one-hot into LSTM (or for that matter sacrifice first dense layer, in any network that will end up using the embedding) and call it a day?

Why would I actually spend time pre-computing Word2Vec embeddings? Yes, the resulting embedings are vectors that are clustered together if the words have similar meaning. But I presume a feed-forward classifier would figure out a good internal model anyway?

$\endgroup$

1 Answer 1

1
$\begingroup$

You don't need word embeddings. Actually, in neural machine translation is frequent not to use them and simply train the embeddings along with the task.

Nevertheless, word embeddings work as a data augmentation technique, as you normally use a different (and much larger) dataset to train them, so they can be useful when you don't have much training data.

Therefore, the decision of using pre-trained word embeddings or not should be driven by the available data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.