I'm working on a project and I created doc2vec representation of different academics which include their patents and publications etc. For each publication and patent I have information such as title and abstract. Now, I want to do a search on all of the professors and find which professor is the most similar to a query string, such as "deep learning" or "computer networking". I have tried to use the infer_vector() to create a doc2vec representation of the query string using the already generated model and calculate the cosine similarity between the vectors. But I got terrible results. For example, when I search for "computer networking", it will give me the result of professor from History. Is there any recommendation of how to find most similar document to a query string?

  • $\begingroup$ Welcome to this site! An easy-to-check alternative would be to use Euclidean distance instead of cosine similarity. $\endgroup$
    – Esmailian
    Mar 30, 2019 at 19:57
  • $\begingroup$ Hm, I tried the euclidean distance, but it gave me similar results to the cosine similarity methods I tried. Is it possible that my query string is too short to give good results? $\endgroup$
    – qiqi
    Mar 30, 2019 at 23:15
  • $\begingroup$ You can go for an easier-to-pass evaluation to see how far off is the model. To this end, see if a good result shows up in top 3, 5, 10 closest matches. Also, keywords in a paper are way more important than the abstract according to your queries, place a special emphasis on them. $\endgroup$
    – Esmailian
    Mar 31, 2019 at 7:58


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.