Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
if Word2Vec is nothing but a transformation of one-hot into a dense vector, why can't I just feed one-hot into LSTM (or for that matter sacrifice first dense layer, in any network that will end up using the embedding) and call it a day?
Why would I actually spend time pre-computing Word2Vec embeddings? Yes, the resulting embedings are vectors that are clustered together if the words have similar meaning. But I presume a feed-forward classifier would figure out a good internal model anyway?
You don't need word embeddings. Actually, in neural machine translation is frequent not to use them and simply train the embeddings along with the task.
Nevertheless, word embeddings work as a data augmentation technique, as you normally use a different (and much larger) dataset to train them, so they can be useful when you don't have much training data.
Therefore, the decision of using pre-trained word embeddings or not should be driven by the available data.
