I recently came across the terms Word2Vec, Sentence2Vec and Doc2Vec and kind of confused as I am new to vector semantics. Can someone please elaborate the differences in these methods in simple words. What are the most suitable tasks for each method?

up vote 11 down vote accepted

Well the names are pretty straight-forward and should give you a clear idea of vector representations.

The Word2Vec Algorithm builds distributed semantic representation of words. There are two main approaches to training, Distributed Bag of Words and The skip gram model. One involves predicting the context words using a centre word, while the other involves predicting the word using the context words. You can read about it in much detail in Mikolov's paper.

The same idea can be extended to sentences and complete documents where instead of learning feature representations for words, you learn it for sentences or documents. However, to get a general idea of a SentenceToVec, think of it as a mathematical average of the word vector representations of all the words in the sentence. You can get a very good approximation just by averaging and without training any SentenceToVec but of course, it has its limitations.

Doc2Vec extends the idea of SentenceToVec or rather Word2Vec because sentences can also be considered as documents. The idea of training remains similar. You can read Mikolov's Doc2Vec paper for more details.

Coming to the applications, it would depend on the task. A Word2Vec effectively captures semantic relations between words hence can be used to calculate word similarities or fed as features to various NLP tasks such as sentiment analysis etc. However words can only capture so much, there are times when you need relationships between sentences and documents and not just words. For example, if you are trying to figure out, whether two stack overflow questions are duplicates of each other.

A simple google search will lead you to a number of applications of these algorithms.

  • What is the difference between averaging word vectors and using doc2vec, then? Does doc2vec account for a word's surroundings in the sentence while building the vector (while the word2vec doesn't)? – John Strood Oct 4 at 12:39
  • Doc2Vec learns a randomly initialized vector for the document along with the words, (document could be a sentence). Manually averaging word vectors doesnt perform to the same capacity because it fails to learn from the entire document. Recently Paragram vectors have been extremely use while working with document similarity etc. – Himanshu Rai Oct 5 at 12:33

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.