I am using gensim library to compute similarity between documents but it only uses cosine similarity. I was wondering if there was a way to use jaccard similarity or any other similarity measure for that matter instead
Is it possible to use Jaccard similarity instead of Cosine similarity in gensim document similarity?
1 Answer
If you have trained a gensim model, that object can act as a dictionary to give you the vector projection (via https://radimrehurek.com/gensim/models/word2vec.html)
$ model['computer'] # raw numpy vector of a word
array([-0.00449447, -0.00310097, 0.02421786, ...], dtype=float32)
So it is possible to manually implement any vector comparison that you choose. Cosign similarity is typically chosen because it performs relatively well when compared to other methods of grouping high dimensional projections.
The way I could envision implementing Jaccard Similarity would be to identify a list of key words on a per document basis, and when comparing document, include words that are synonyms as intersections.
Based on reviewing the gensim document comparison text (https://radimrehurek.com/gensim/similarities/docsim.html) I don't believe there is a native implementation.