12 votes

What is Hellinger Distance and when to use it?

Hellinger distance is a metric to measure the difference between two probability distributions. It is the probabilistic analog of Euclidean distance. Given two probability distributions, $P$ and $Q$, ...
11 votes

When would one use Manhattan distance as opposed to Euclidean distance?

According to this interesting paper, Manhattan distance (L1 norm) may be preferable to Euclidean distance (L2 norm) for the case of high dimensional data. The authors of the paper even go a step ...
  • 1,707
11 votes
Accepted

Why is the cosine distance used to measure the similatiry between word embeddings?

You're asking two questions here. Does this mean the magnitude of the vectors is irrelevant? Yes. Cosine distance is $ D_{cos} = \frac{A \cdot B}{\|A\|\|B\|} $, which just comes from the definition ...
  • 1,174
10 votes
Accepted

Cosine Distance > 1 in scipy

The cosine distance formula is: And the formula used by the cosine function of the spatial class of scipy is: So, the ...
  • 8,116
10 votes
Accepted

How to evaluate the K-Modes Clusters?

The are some techniques to choose the number of clusters K. The most common ones are The Elbow Method and The Silhouette Method. Elbow Method In this method, you ...
8 votes
Accepted

What methods exist for distance calculation in clustering? when we should use each of them?

Well, there is a book called Deza, Michel Marie, and Elena Deza. Encyclopedia of distances. Springer Berlin Heidelberg, 2009. ISBN 978-3-642-00233-5 I guess that book answers your question better ...
7 votes
Accepted

Coordinate System's influence on $L$ distances (Manhattan and Euclidean)

For example, consider the green line. What is its length? In $L_2$, the answer is $1$, in $L_1$, the answer is $1$ as well. Now, for the same line, let's rotate it $45^\circ$ counterclockwise. What ...
7 votes

Levenshtein distance vs simple for loop

When I understand you correctly, you would loop over a string character by character and compare if there is a match at the same position in some other string. Drawing from this post, you find that: <...
  • 7,044
5 votes
Accepted

How to evaluate distance in k-means clusters?

There are several important points to keep in mind in considering your questions: You should always normalize or standardize your data before applying k-means clustering. This is true of most other ...
  • 6,738
5 votes

When would one use Manhattan distance as opposed to Euclidean distance?

I found something which might be intuition about this problem in Hands-On Machine Learning with Scikit-Learn and TensorFlow Both the RMSE and the MAE are ways to measure the distance between two ...
5 votes

When would one use Manhattan distance as opposed to Euclidean distance?

I can suggest a couple ideas, from wikipedia. If you want to place less emphasis on outliers, manhattan distance will try to reduce all errors equally since the gradient has constant magnitude. If ...
5 votes
Accepted

How the squared Euclidean distance is an example of non-metric function?

Let $x, y \in \mathbb{R}^n$. The Euclidean distance $d$ is defined as $$ d(x,y) = \sqrt{\sum_{i=1}^n (x_i - y_i)^2}. $$ The squared Euclidean distance is therefore $$ d(x,y)^2 = \sum_{i=1}^n (x_i - ...
5 votes
Accepted

Levenshtein distance vs simple for loop

A simple example shows the difference: let's compare the strings hello! and hhello!: The Levenshtein distance finds that the ...
  • 23.8k
4 votes
Accepted

Is there a way to measure correlation between two similar datasets?

I see a lot of people post this similar question on StackExchange, and the truth is that there is no methodology to compare if data set A looks like set B. You can compare summary statistics, such as ...
  • 481
4 votes

Is there a way to measure correlation between two similar datasets?

I would take a look at Canonical correlation Analysis.
  • 1,305
4 votes

When would one use Manhattan distance as opposed to Euclidean distance?

The use of Manhattan distance depends a lot on the kind of co-ordinate system that your dataset is using. While Euclidean distance gives the shortest or minimum distance between two points, Manhattan ...
4 votes

Statistical distances for time series of distributions

This is similar to the fundamental information theory problem that Shannon explored. In that domain, it is framed this way: given two rvs, X and Y, what information does X convey about Y? An example ...
4 votes

Definition - center of the cluster with non-Euclidean distance

The easiest solution for a non-Euclidean cluster center is the medoid, as in the algorithm PAM. This works with arbitrary metrics, but unfortunately for a 2-element cluster the result by definition is ...
4 votes
Accepted

Improve k-means accuracy

Great question, @gsamaras! The way you've set up this experiment makes a lot of sense to me, from a design point of view, but I think there are a couple aspects you can still examine. First, it's ...
  • 1,473
4 votes

Clustering algorithm for a distance matrix

Within the documentation for HDBSCAN (Hierarchical DBSCAN), there is a really nice comparison of clustering algorithms. It is a bit biased, highlighting its own strengths (of course), but will still ...
  • 14.2k
4 votes

Distance between users

No need for algorithms, or recommendation systems. You have: For each user a have a bunch of features. As long as they're numeric, or can be made numeric (e.g. aggregating the values or one-hot-...
  • 482
4 votes
Accepted

How do I test a difference between two proportions representing fatality rate for Covid 19 in Philippines and World (except Philippines)?

It is not a case of paired nominal data. Hence, Mc Nemar's test can not be applied to check whether there is a higher fatality rate in Philippines ?. THE fatality rate is given for Philippines and ...
3 votes

Dimension reduction techniques in R that do not use the full distance matrix

You can try using an auto-encoder which also is a non-linear dimension reduction technique. It uses a neural network framework to find the most efficient transformation from $p$ dimensions down to ...
3 votes

How to build a mean prototype from data

The object with the lowest mean distance (= the object with the lowest distance sum) is known as the medoid and the basis of k-medoids algorithms such as PAM. Because you must not use k-means with ...
3 votes
Accepted

Fixing data inconsistencies

Since this dataset is already organised in a table, you can leverage standard SQL functions to perform a large part of the cleanup. A record seems to be composed of 4 fields, for example: ...
3 votes
Accepted

Alternative distance to Dynamic Time Warping

Like suggested in one answer on this SO question, you could use elastic matching with Levenshtein distance to your task. Levenshtein distance obeys triangle inequality and is therefore a metric ...
  • 569
3 votes
Accepted

Knn and euclidean distance

You can think of examples as vectors in $\mathbb{R}^p$, where $p$ is the number of features. Two examples will be very similar if the distance between them is close to $0$ (in the extreme case, if two ...
  • 5,859
3 votes

How to calculate the silhouette coefficient?

a(i) : the average distance between 'i' and all other data within the same cluster (source) b(i) : the lowest average distance of 'i' to all points in any other cluster, of which 'i' is not a member ...
  • 1,187
3 votes

Euclidean distance for more than two datapoints

You can use sklearn's euclidean_distances function. http://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.euclidean_distances.html#sklearn....
  • 435
3 votes

Clustering algorithm for a distance matrix

There are hundreds of algorithms to choose from. Hierarchical clustering in it's myriad of variants. Cut the dendrogram as desired, e.g., to get k clusters PAM, the closest match to k-means on a ...

Only top scored, non community-wiki answers of a minimum length are eligible