5
$\begingroup$

I have set of categories and I want to compare a document vector with word vector of categories to find best matching category.

Is it possible to compare a word vector with document vector? If yes, is there any literature which gives proof of concept for this?

$\endgroup$
10
$\begingroup$

In paragraph vector, the vector tries to grasp the semantic meaning of all the words in the context by placing the vector itself in each and every context. Thus finally, the paragraph vector contains the semantic meaning of all the words in the context trained.

When we compare this to word2vec, each word in word2vec preserves its own semantic meaning. Thus summing up all the vectors or averaging them will result in a vector which could have all the semantics preserved. This is sensible, because when we add the vectors (transport+water) the result nearly equals ship or boat, which means summing the vectors sums up the semantics.

Before the paragraph vector paper got published, people used averaged word vectors as sentence vectors. To be honest, in my work these average vectors work better than document vectors. So, with these things in mind, in this way it could be compared.

$\endgroup$
  • 1
    $\begingroup$ Hi, Can you explain " the paragraph vector contains the semantic meaning of all the words in the context trained". Any idea, how are all these word-vectors combined to form a single paragraph-vector? $\endgroup$ – Vishwa Sep 6 '16 at 8:46
  • $\begingroup$ @Vishwa: Its based on distributional hypothesis. The semantic meaning of a word is conveyed by its surrounding context words. For more detail read the original paper, Distributed Representations of Words and Phrases and their Compositionality by Thomas Mikolov et al. $\endgroup$ – yazhi Sep 9 '16 at 18:58
  • $\begingroup$ Thanks a lot for your great answer. I am interested in knowing how you summed or averaged the vectors because I would like to try it for my dataset as well. Did you average all the word vectors in the document or only selected word vectors? $\endgroup$ – Volka Oct 17 '17 at 0:46
  • 1
    $\begingroup$ @Volka In the pre-processing steps, I removed stop words, numbers and symbols. Summed up the rest of the vectors. Its also possible to apply tf-idf to find out the word importance and select words based on the tf-idf values. $\endgroup$ – yazhi Oct 17 '17 at 9:28
  • 1
    $\begingroup$ Apply tf-idf on multiple documents to get which words really contribute to each document. ref: tfidf.com $\endgroup$ – yazhi Oct 17 '17 at 10:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.