42
votes
When to use cosine simlarity over Euclidean similarity
When to use cosine similarity over Euclidean similarity
Cosine similarity looks at the angle between two vectors, euclidian similarity at the distance between two points.
Let's say you are in an e-...
- 19.1k
41
votes
Accepted
Adaboost vs Gradient Boosting
Both AdaBoost and Gradient Boosting build weak learners in a sequential fashion.
Originally, AdaBoost was designed in such a way that at every step the sample distribution was adapted to put more ...
- 6,422
40
votes
Accepted
Sentence similarity prediction
Your problem can be solved with Word2vec as well as Doc2vec. Doc2vec would give better results because it takes sentences into account while training the model.
Doc2vec solution
You can train your ...
- 706
20
votes
Accepted
How to measure the similarity between two images?
Check this handout!
Well, there a few so... lets go:
Given two images $J[x,y]$ and $I[x,y]$ with $(x,y) \in N^{N \times M}$...
A - Used in template matching:
Template Matching is linear and is not ...
- 1,676
18
votes
Best practical algorithm for sentence similarity
Cosine Similarity for Vector Space could be you answer.
Or you could calculate the eigenvector of each sentences. But the Problem is, what is similarity?
"This is a tree",
"This is not ...
- 351
14
votes
When to use cosine simlarity over Euclidean similarity
Ok, so, your intuition here is wrong. Not necessarily about the examples you gave, but the fact that you think Euclidian distance could be useful in 200 dimensional space. 200d space is so, so empty. ...
- 261
14
votes
Accepted
Why use cosine similarity instead of scaling the vectors when calculating the similarity of vectors?
Let $u, v$ be vectors. The "cosine distance" between them is given by
$$d_{\cos}(u, v) = 1 - \frac{u}{\|u\|} \cdot \frac{v}{\|v\|} = 1 - \cos \theta_{u,v},$$
and the proposed "...
- 270
13
votes
Accepted
Cosine similarity vs The Levenshtein distance
As mentioned in other answers, traditionally cosine is used to measure similarity between vectors whereas Levenshtein is used as a string similarity measure, i.e. measuring the distance between ...
- 25.7k
10
votes
Sentence similarity prediction
Word Mover’s Distance (WMD) is an algorithm for finding the distance between sentences. WMD is based on word embeddings (e.g., word2vec) which encode the semantic meaning of words into dense vectors.
...
- 22.1k
9
votes
Best practical algorithm for sentence similarity
One approach you could try is averaging word vectors generated by word embedding algorithms (word2vec, glove, etc). These algorithms create a vector for each word and the cosine similarity among them ...
- 2,226
8
votes
Applications and differences for Jaccard similarity and Cosine Similarity
The answer by saq7 is wrong.
Where $\mathbf{a}$ and $\mathbf{b}$ are binary vectors, they can be interpreted as sets of indices with value 1. Let's therefore consider sets $A$ and $B$.
Jaccard ...
8
votes
Accepted
Text similarity with sentence embeddings
One approach is using Word Mover’s Distance (WMD). WMD is an algorithm for finding the distance between texts of different lengths, where each word is represented as a word embedding vector.
The ...
- 22.1k
7
votes
Accepted
How to measure the similarity between two text documents?
In general,there are two ways for finding document-document similarity
TF-IDF approach
Make a text corpus containing all words of documents . You have to use tokenisation and stop word removal . ...
- 116
7
votes
Similarity of words using BERTMODEL
First of all, I think you are confused with pretrained and finetuned.
BERT is pretrained on ...
- 1,004
5
votes
Accepted
How to compute the Jaccard Similarity in this example? (Jaccard vs. Cosine)
Cosine similarity is for comparing two real-valued vectors, but Jaccard similarity is for comparing two binary vectors (sets). So you cannot compute the standard Jaccard similarity index between your ...
- 3,940
5
votes
Sentence similarity prediction
You can try an easy solution using sklearn and it's going to work fine.
Use tfidfvectorizer to get a vector representation of each text
Fit the vectorizer with your data, removing stop-words.
...
- 760
5
votes
Calculate similarity on boolean data
You should look at the Jaccard Index, is the de facto similarity between set of items, where the sets are represented using a boolean vector. In this boolean vector each coordinate represents an item, ...
- 2,226
5
votes
When to use cosine simlarity over Euclidean similarity
The intuition built by the top response is spot-on for tf-idf vectors, and carries over to any vector that naturally wants to be normalized. However, in such circumstances, cosine similarity is ...
5
votes
Similarity score: Can Sklearn SVR predict values greater than 1 and less than 0?
I am using svm.SVR() from scikit-learn to apply Logistic Regression on my training data to solve similarity problem.
Wait a second, if you're using support-vector regression, then you're not using ...
- 2,764
5
votes
Cosine similarity vs The Levenshtein distance
I think the answers you got are technically correct, but don't address the big picture.
In the data science world, cosine similarity is mainly used for documents which have been encoded by an ...
- 326
5
votes
Why use cosine similarity instead of scaling the vectors when calculating the similarity of vectors?
In your example, User 1 and User 2 bought the same ingredients, but User 2 bought 100x more ingredients than User 1. If you normalize and use Euclidean distance, then the distance is 0 (by the ...
- 669
4
votes
Accepted
Is there a way to measure correlation between two similar datasets?
I see a lot of people post this similar question on StackExchange, and the truth is that there is no methodology to compare if data set A looks like set B. You can compare summary statistics, such as ...
- 481
4
votes
Is there a way to measure correlation between two similar datasets?
I would take a look at Canonical correlation Analysis.
- 1,337
4
votes
Accepted
How to find similar time series?
Since the time-series are annual, the data points you have for each time-series are limited and also quite distant (the values are 1 year apart). So I wouldn't use Dynamic Time Wrapping on your data.
...
- 576
4
votes
Can cosine similarity be applied to multidimensional matrices?
Don't hack. Do the math instead.
You could reshape your matrix into a vector, then use cosine.
But whether that is sensible to do: ask yourself.
You could also ignore the matrix and always return 0....
- 8,064
4
votes
Distance between users
No need for algorithms, or recommendation systems. You have:
For each user a have a bunch of features.
As long as they're numeric, or can be made numeric (e.g. aggregating the values or one-hot-...
- 482
4
votes
Accepted
Comparing one small dataset with a big dataset for similar records
A way to speed up this process is to preprocess the large dataset, the goal being to store the documents from A in a way which avoids a lot of useless comparisons.
Store each document from A in an ...
- 25.7k
4
votes
Accepted
Has bloom filters a higher probability of collide when strings are similar
It depends on the hash function but in general no, because standard hash functions are designed to avoid that similar objects have similar hash codes. [edited, see comment]
For your use case you ...
- 25.7k
4
votes
Accepted
Higher level sentence similarity (meaning instead of 'just' embeddings)
The similarity used to train this model might be different from the similarity you expect.
A better approach would be create your own large and good quality training set of similar and dissimilar ...
- 1,254
3
votes
Text similarity using RNN
Doc2Vec, Mikolov's paper will solve your problem. Here is the paper. You can find a gensim implementationhere. While using RNN, using GLOVE or Googl Word2Vec will be always useful even if your ...
- 1,858
Only top scored, non community-wiki answers of a minimum length are eligible