2
$\begingroup$

Like Word2vec is not a single algorithm but combination of two, namely, CBOW and Skip-Gram model; is Doc2Vec also a combination of any such algorithms? Or is it an algorithm in itself?

$\endgroup$
3
$\begingroup$

Word2Vec is not a combination of two models, rather both are variants of word2vec. Similarly doc2vec has Distributed Memory(DM) model and Distributed Bag of words (DBOW) model. Based on the context words and the target word, these variants arised.

Note: the name of the model maybe confusing

  • Distriubted Bag of words is similar to Skip-gram model
  • Distributed Memory is similar to Continuous bag of words model
$\endgroup$
1
$\begingroup$

Documentation says:

Deep learning via the distributed memory and distributed bag of words models from [1], using either hierarchical softmax or negative sampling [2], [3].

$\endgroup$
  • 1
    $\begingroup$ Those are algorithms used for approximation. and you could use hierarchical softmax or negative sampling in any models, CBOW or SG. $\endgroup$ – yazhi Jul 10 '17 at 11:43
  • $\begingroup$ Are DM and DBOW deep learning? I have read CBOW and SG uses a shallow neural network to predict word/context. Shouldn't it be the same for Doc2Vec? $\endgroup$ – Kshitiz Jul 10 '17 at 19:28
  • $\begingroup$ doc2vec uses the same algorithm as word2vec, except that it uses sentence labels. None of the models are deep learning to be specific. As deep learning model should have multiple hidden layers. Since, deep learning is mostly involved with big data and word2vec and doc2vec mostly works with big data, some term it as deep learning model. But actually it is not. $\endgroup$ – yazhi Jul 11 '17 at 7:29
1
$\begingroup$

Distributed Memory model preserves the word order in a document whereas Distributed Bag of words just uses the bag of words approach, which doesn't preserve any word order.

This has been explained in details in this research paper.

$\endgroup$
  • $\begingroup$ No problem, and welcome to Data Science! $\endgroup$ – Stephen Rauch Mar 21 '18 at 4:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.