I am creating a Doc2Vec model out of hundreds of PDF documents.
I have 17 documents that are part of this Doc2Vec that I want to use to check similarity with other documents in the Doc2Vec model.
For instance I want to something like: model.similarity(tag5, tag30)
Can this be done?
Based on what you have shared, it seems like Doc2Vec should be suited to your objective.
That said, I think the Doc in the package name can lead people astray. It gives the impression that you can feed in any documents and it will find the similarities among them. And while you certainly can feed whole documents, the reality is that the similarity / dissimilarity may not be aligned in a useful way to your task. I have found that training the model on smaller chunks of the document, like sentences or paragraphs, allows for the model to identify subtle differences that can then be aggregated to the whole document.
In your case, are the 17 documents you mentioned, exemplars of 17 different types of documents whose label you want to assign to the entire corpus? If so, Doc2Vec may be overkill. Have you looked into using Tf-Idf? Sometimes this approach works well if you don't necessarily need the word embeddings. Just some food for thought.
Yes there multiples ways to do it. Simple ones TF-IDF, SIF and quick/skip thought use encoder-decoder structure and the output of encoder is the embedding. Then the similarity between documents is simply the cos of embeddings. Doc2vec ultimately generate embeddings too.
I think you can find this answer a great solution to your problem. I've used it successfully in my case.
