19
$\begingroup$

I am wondering how to label (tag) sentences / paragraphs / documents with doc2vec in gensim - from a practical standpoint.

Do you need to have each sentence / paragraph / document with its own unique label (e.g. "Sent_123")? This seems useful if you want to say "what words or sentences are most similar to a single specific sentence labeled "Sent_123".

Can you have the labels be repeated based on content? For example if each sentence / paragraph / document is about a certain product item (and there are multiple sentence / paragraph / document for a given product item) can you label the sentences based on the item and then compute the similarity between a word or a sentence and this label (which I guess would be like an average of all those sentences that had to do with the product item)?

$\endgroup$
12
$\begingroup$

Both are possible. You can give every document a unique ID (such as a sequential serial number) as a doctag, or a shared string doctag representing something else about it, or both at the same time.

The TaggedDocument constructor takes a list of tags. (If you happen to limit yourself to to plain ints ascending from 0, the Doc2Vec model will use those as direct indexes into its backing array, and you'll save a lot of memory that would otherwise be devoted to a string -> index lookup, which could be important for large datasets. But you can use string doctags or even a mixture of int and string doctags.)

You'll have to experiment with what works best for your needs.

For some classification tasks, an approach that's sometimes worked better than I would have expected is skipping per-text IDs entirely, and just training the Doc2Vec model with known-class examples, with the desired classes as the doctags. You then get 'doc vectors' just for the class doctags – not every document – a potentially much smaller model. Later inferring vectors for new texts results in vectors meaningfully close to related class doc vectors.

| improve this answer | |
$\endgroup$
  • $\begingroup$ That's great thanks very much! As an aside as I am starting to play around with genim - I am curious if it is possible to calculate the similarity between a doc (regardless if unique or shared tags are used ) and a word - can that be done in your experience? $\endgroup$ – B_Miner Feb 13 '16 at 2:26
  • 1
    $\begingroup$ Some training modes create word and document vectors inside the 'same space' and so similarities may be meaningful. See for example this paper – arxiv.org/abs/1507.07998 – which even does a sort of 'analogy arithmetic' involving doc-vectors (Wikipedia articles) and word-vectors. The training in that paper is like gensim's DBOW mode with concurrent skip-gram word-training: dm=0, dbow_words=1. $\endgroup$ – gojomo Feb 13 '16 at 10:01
  • $\begingroup$ Thanks @gojomo! I added a query about how to do this on the gensim list: groups.google.com/forum/#!topic/gensim/RLRfY6k3ulw $\endgroup$ – B_Miner Feb 13 '16 at 14:03
  • $\begingroup$ Please correct me if I am wrong. In your last paragraph, you are suggesting that each document should be tagged like we tag questions in this site. And after training, we will get vector representation of each tag. When a new document arrive, we can just use similarity metric for suggesting tags for the new document. $\endgroup$ – user May 24 '16 at 2:05
  • $\begingroup$ @user - yes, it is possible and sometimes beneficial to do that, though it's not necessary. $\endgroup$ – gojomo May 24 '16 at 21:52
10
$\begingroup$

doc2vec model gets its algorithm from word2vec.

In word2vec there is no need to label the words, because every word has their own semantic meaning in the vocabulary. But in case of doc2vec, there is a need to specify that how many number of words or sentences convey a semantic meaning, so that the algorithm could identify it as a single entity. For this reason, we are specifying labels or tags to sentence or paragraph depending on the level of semantic meaning conveyed.

If we specify a single label to multiple sentences in a paragraph, it means that all the sentences in the paragraph are required to convey the meaning. On the other hand, if we specify variable labels to all the sentences in a paragraph, it means that each conveys a semantic meaning and they may or may not have similarity among them.

In simple terms, a label means semantic meaning of something.

| improve this answer | |
$\endgroup$
  • $\begingroup$ If we specify a single label to multiple sentences in a paragraph, it means that all the sentences in the paragraph are required to convey the meaning. I am not sure I am understanding this correctly. From the algorithms POV, is it all the sentences with same tag are needed for the semantic definition or all the sentences with the same tag describe same thing? In first case, no single sentence is self-sufficient by itself, in the second case, a single sentence is self-sufficient. $\endgroup$ – user May 24 '16 at 1:59
  • $\begingroup$ @user: It is the second case, from the algorithm POV, a tag is defined as an entity which when used in a single sentence, it captures the meaning of all the words. In the same way when used in multiple sentences it also captures the words in all other sentences. $\endgroup$ – yazhi May 24 '16 at 10:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.