I'm working on a project and I created doc2vec representation of different academics which include their patents and publications etc. For each publication and patent I have information such as title and abstract. Now, I want to do a search on all of the professors and find which professor is the most similar to a query string, such as "deep learning" or "computer networking". I have tried to use the infer_vector() to create a doc2vec representation of the query string using the already generated model and calculate the cosine similarity between the vectors. But I got terrible results. For example, when I search for "computer networking", it will give me the result of professor from History. Is there any recommendation of how to find most similar document to a query string?
3
-
$\begingroup$ Welcome to this site! An easy-to-check alternative would be to use Euclidean distance instead of cosine similarity. $\endgroup$– EsmailianMar 30, 2019 at 19:57
-
$\begingroup$ Hm, I tried the euclidean distance, but it gave me similar results to the cosine similarity methods I tried. Is it possible that my query string is too short to give good results? $\endgroup$– qiqiMar 30, 2019 at 23:15
-
$\begingroup$ You can go for an easier-to-pass evaluation to see how far off is the model. To this end, see if a good result shows up in top 3, 5, 10 closest matches. Also, keywords in a paper are way more important than the abstract according to your queries, place a special emphasis on them. $\endgroup$– EsmailianMar 31, 2019 at 7:58
Add a comment
|