Share Your Experience: Take the 2024 Developer Survey

# Tag Info

Accepted

### Why is the cosine distance used to measure the similatiry between word embeddings?

You're asking two questions here. Does this mean the magnitude of the vectors is irrelevant? Yes. Cosine similarity is $S_{cos} = \frac{A \cdot B}{\|A\|\|B\|}$, which just comes from the ...
• 1,284

### What is Hellinger Distance and when to use it?

Hellinger distance is a metric to measure the difference between two probability distributions. It is the probabilistic analog of Euclidean distance. Given two probability distributions, $P$ and $Q$, ...
• 21.5k

### When would one use Manhattan distance as opposed to Euclidean distance?

According to this interesting paper, Manhattan distance (L1 norm) may be preferable to Euclidean distance (L2 norm) for the case of high dimensional data. The authors of the paper even go a step ...
• 1,787
Accepted

### How to evaluate the K-Modes Clusters?

The are some techniques to choose the number of clusters K. The most common ones are The Elbow Method and The Silhouette Method. Elbow Method In this method, you ...
• 3,548
Accepted

### What methods exist for distance calculation in clustering? when we should use each of them?

Well, there is a book called Deza, Michel Marie, and Elena Deza. Encyclopedia of distances. Springer Berlin Heidelberg, 2009. ISBN 978-3-642-00233-5 I guess that book answers your question better ...
Accepted

### Coordinate System's influence on $L$ distances (Manhattan and Euclidean)

For example, consider the green line. What is its length? In $L_2$, the answer is $1$, in $L_1$, the answer is $1$ as well. Now, for the same line, let's rotate it $45^\circ$ counterclockwise. What ...
• 3,108

### When would one use Manhattan distance as opposed to Euclidean distance?

I found something which might be intuition about this problem in Hands-On Machine Learning with Scikit-Learn and TensorFlow Both the RMSE and the MAE are ways to measure the distance between two ...

### Levenshtein distance vs simple for loop

When I understand you correctly, you would loop over a string character by character and compare if there is a match at the same position in some other string. Drawing from this post, you find that: <...
• 7,516
Accepted

### How to evaluate distance in k-means clusters?

There are several important points to keep in mind in considering your questions: You should always normalize or standardize your data before applying k-means clustering. This is true of most other ...
• 6,818

### When would one use Manhattan distance as opposed to Euclidean distance?

I can suggest a couple ideas, from wikipedia. If you want to place less emphasis on outliers, manhattan distance will try to reduce all errors equally since the gradient has constant magnitude. If ...
• 191
Accepted

### How the squared Euclidean distance is an example of non-metric function?

Let $x, y \in \mathbb{R}^n$. The Euclidean distance $d$ is defined as $$d(x,y) = \sqrt{\sum_{i=1}^n (x_i - y_i)^2}.$$ The squared Euclidean distance is therefore  d(x,y)^2 = \sum_{i=1}^n (x_i - ...
• 1,646
Accepted

### Levenshtein distance vs simple for loop

A simple example shows the difference: let's compare the strings hello! and hhello!: The Levenshtein distance finds that the ...
• 25.5k

### When would one use Manhattan distance as opposed to Euclidean distance?

The use of Manhattan distance depends a lot on the kind of co-ordinate system that your dataset is using. While Euclidean distance gives the shortest or minimum distance between two points, Manhattan ...
• 141
Accepted

### Is there a way to measure correlation between two similar datasets?

I see a lot of people post this similar question on StackExchange, and the truth is that there is no methodology to compare if data set A looks like set B. You can compare summary statistics, such as ...
• 481

### Is there a way to measure correlation between two similar datasets?

I would take a look at Canonical correlation Analysis.
• 1,337

### Clustering algorithm for a distance matrix

Within the documentation for HDBSCAN (Hierarchical DBSCAN), there is a really nice comparison of clustering algorithms. It is a bit biased, highlighting its own strengths (of course), but will still ...
• 14.9k

### Distance between users

No need for algorithms, or recommendation systems. You have: For each user a have a bunch of features. As long as they're numeric, or can be made numeric (e.g. aggregating the values or one-hot-...
• 482
Accepted

### How do I test a difference between two proportions representing fatality rate for Covid 19 in Philippines and World (except Philippines)?

It is not a case of paired nominal data. Hence, Mc Nemar's test can not be applied to check whether there is a higher fatality rate in Philippines ?. THE fatality rate is given for Philippines and ...
Accepted

### Fixing data inconsistencies

Since this dataset is already organised in a table, you can leverage standard SQL functions to perform a large part of the cleanup. A record seems to be composed of 4 fields, for example: ...
• 2,537

### Dimension reduction techniques in R that do not use the full distance matrix

You can try using an auto-encoder which also is a non-linear dimension reduction technique. It uses a neural network framework to find the most efficient transformation from $p$ dimensions down to ...
• 883
Accepted

### Alternative distance to Dynamic Time Warping

Like suggested in one answer on this SO question, you could use elastic matching with Levenshtein distance to your task. Levenshtein distance obeys triangle inequality and is therefore a metric ...
• 569
Accepted

### Knn and euclidean distance

You can think of examples as vectors in $\mathbb{R}^p$, where $p$ is the number of features. Two examples will be very similar if the distance between them is close to $0$ (in the extreme case, if two ...
• 6,081

### How to calculate the silhouette coefficient?

a(i) : the average distance between 'i' and all other data within the same cluster (source) b(i) : the lowest average distance of 'i' to all points in any other cluster, of which 'i' is not a member ...
• 1,282

### Euclidean distance for more than two datapoints

You can use sklearn's euclidean_distances function. http://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.euclidean_distances.html#sklearn....
• 435

### Clustering algorithm for a distance matrix

There are hundreds of algorithms to choose from. Hierarchical clustering in it's myriad of variants. Cut the dendrogram as desired, e.g., to get k clusters PAM, the closest match to k-means on a ...
Accepted

### Are there algorithms for clustering objects with pairwise distances, without computing all pairwise distances?

Well, one may argue that DBSCAN is based on all pairwise distances, but it uses data indexing to avoid computing all of them using geometric bounds. And there are other examples if you browse through ...

### Is the distance in Nearest Neighbors a good measure of similarity?

This is counter-intuitive, because one would expect [0,2,0,0] to be more similar to [0,1,0,0] than [0,1,1,0]. No this is expected, since the two points are exactly at the same distance in the ...
• 25.5k

### Clustering algorithm which does not require to tell the number of clusters

I would try DBSCAN algorithm first: fairly easy to tune (with, in particular, a notion of distance as you requested), and does not need to know the number of clusters. There are a few other ...
• 1,327
Accepted

### Similarity Measure between two feature vectors

So, thank you for clarifying the question. Just to confirm that the question is asking how to set an appropriate threshold for face feature vectors (represented a a ...
• 1,438