Word2Vec vs. Sentence2Vec vs. Doc2Vec

Question

I recently came across the terms Word2Vec, Sentence2Vec and Doc2Vec and kind of confused as I am new to vector semantics. Can someone please elaborate the differences in these methods in simple words. What are the most suitable tasks for each method?

Answer 1

Well the names are pretty straight-forward and should give you a clear idea of vector representations.

The Word2Vec Algorithm builds distributed semantic representation of words. There are two main approaches to training, Continuous Bag of Words and The skip gram model. One involves predicting the context words using a centre word, while the other involves predicting the word using the context words. You can read about it in much detail in Mikolov's paper.

The same idea can be extended to sentences and complete documents where instead of learning feature representations for words, you learn it for sentences or documents. However, to get a general idea of a SentenceToVec, think of it as a mathematical average of the word vector representations of all the words in the sentence. You can get a very good approximation just by averaging and without training any SentenceToVec but of course, it has its limitations.

Doc2Vec extends the idea of SentenceToVec or rather Word2Vec because sentences can also be considered as documents. The idea of training remains similar. You can read Mikolov's Doc2Vec paper for more details.

Coming to the applications, it would depend on the task. A Word2Vec effectively captures semantic relations between words hence can be used to calculate word similarities or fed as features to various NLP tasks such as sentiment analysis etc. However words can only capture so much, there are times when you need relationships between sentences and documents and not just words. For example, if you are trying to figure out, whether two stack overflow questions are duplicates of each other.

A simple google search will lead you to a number of applications of these algorithms.