# Questions tagged [distance]

For question regarding distance between distributions or variables, such as Euclidean distance between points in n-space.

126 questions
Filter by
Sorted by
Tagged with
5answers
42k views

### When would one use Manhattan distance as opposed to Euclidean distance?

I am trying to look for a good argument on why one would use the Manhattan distance over the Euclidean distance in machine learning. The closest thing I found to a good argument so far is on this MIT ...
1answer
12k views

### What is Hellinger Distance and when to use it?

I am interested in knowing what really happens in Hellinger Distance (in simple terms). Furthermore, I am also interested in knowing what are types of problems that we can use Hellinger Distance? ...
1answer
461 views

### Finding linear transformation under which distance matrices are similar

I have $n$ sets of vectors, where each set $S_i$ contains $k$ vectors in $\mathbb{R}^d$. I know there is some unknown linear transformation $W$ under which the distance matrix $D_i$ (a $k\times k$ ...
2answers
669 views

### Fixing data inconsistencies

I'm trying to analyze some data I have but there is a lot of inconsistencies in my data. I have a SQL table that I'm trying to analyze. The table is a table of universities with the following ...
5answers
8k views

### Is there a way to measure correlation between two similar datasets?

Let's say that I have two similar datasets with the same size of elements, for example 3D points : Dataset A : { (1,2,3), (2,3,4), (4,2,1) } Dataset B : { (2,1,3), (2,4,6), (8,2,3) } And the ...
1answer
9k views

### Cosine Distance > 1 in scipy

I am working on a recommendation engine, and I have chosen to use SciPy's cosine distance as a way of comparing items. I have two vectors: ...
4answers
617 views

### What methods exist for distance calculation in clustering? when we should use each of them?

What methods exist for distance calculation in clustering? like Manhattan, Euclidean, etc.? Plus, I don't know when I should use them. I always use Euclidean distance.
2answers
104 views

### How do I test a difference between two proportions representing fatality rate for Covid 19 in Philippines and World (except Philippines)?

I'm trying to analyse if the fatality rate from my country (A third world country) vary significantly from the world's fatality rate. So I'd basically have two samples, labeled (Philippines) and (...
1answer
2k views

### Improve k-means accuracy

Our weapons: I am experimenting with k-means and Hadoop, where I am chained to these options for various reasons (e.g. Help me win this war!). The battlefield: I have articles, which belong to c ...
2answers
2k views

### Alternative distance to Dynamic Time Warping

I am performing a comparison among time series by using Dynamic Time Warping (DTW). However, it is not a real distance, but a distance-like quantity, since it doesn't assure the triangle inequality to ...
1answer
923 views

### Why is the cosine distance used to measure the similatiry between word embeddings?

While computing the similarity between the words, cosine similarity or distance is computed on word vectors. Why aren't other distance metrics such as Euclidean ...
2answers
963 views

### Coordinate System's influence on $L$ distances (Manhattan and Euclidean)

I don't understand this picture, which says if we change the coordinate system, we would have the same result for $L_2$ distance, whereas, our result would differ for $L_1$ distance. What does it ...
2answers
3k views

### How the squared Euclidean distance is an example of non-metric function?

I am reading a book on Pattern Recognition (by Prof V Susheela Devi and Prof Murty) where in the chapter of data representation 2.3.3 the non metric similarity function is defined as those which do ...
2answers
857 views

### Statistical distances for time series of distributions

I am interested in clustering $N$ time series of $T$ 'values' each. These values are distributions (which can be represented by their cumulative distribution functions (cdf), or their probability ...
1answer
1k views

### Can I use euclidean distance for Latent Dirichlet Allocation document similarity?

I have a Latent Dirichlet Allocation (LDA) model with $K$ topics trained on a corpus with $M$ documents. Due to my hyper parameter configurations, the output topic distributions for each document is ...
3answers
262 views

### Cluster elements that appear in the same lists

Suppose I have a multitude of sets with (unordered) combinations of elements and I want to determine which elements tend to appear together. For example Given the following sets: ...
6answers
397 views

### Distance between users

I want to compute the "distance" between users in order to return the top n similar users, for any given user. For each user a have a bunch of features. This is close to a recommendation system, ...
3answers
3k views

### Clustering algorithm for a distance matrix

I have a similarity matrix between N objects. For each N objects, I have a measure of how similar they are between each others - 0 being identical (the main diagonal) and increasing values as they get ...
2answers
12k views

### How to calculate the silhouette coefficient?

Calculate the silhouette coefficient of point Pi from the above image. To apply the given formula, how to know which is a(i) and b(i)?
2answers
58 views

### How to build a mean prototype from data

I have a dataset with physiological measures of subjects along time. I would like to create (or select) a mean prototype example in order to be able to identify in new examples how far are they from ...
3answers
171 views

### A metric between trees

I have certain tree structures. I am not an expert in machine learning. As I would with take KNN, I would calculate distances via metric function and a new data point and the points from the training ...
3answers
111 views

### Distance between very large discrete probability distributions

I have 192 countries where each country has some value for 1 million attributes which sum up to 1 (a discrete probability distribution). For any one country most of the values for the attributes are 0....
1answer
81 views

### KNN Regression: Distance function and/or vector representation for datetime features

Context: Trying to forecast some sort of consumption value (e.g. water) using datetime features and exogenous variables (like temperature). Take some datetime features like week days (...
1answer
783 views

### K Nearest Neighbour with different distance matrix to each datapoint

I'm wondering if there is library support in python (such as sklearn) for doing KNN on a data set that has a custom distance matrix (positive definite) for each data point (x is a query point, $x_i$ ...
2answers
6k views

### How to evaluate the K-Modes Clusters?

K-modes algorithm is available here I want to do clustering of my binary dataset. I need to specify the number of clusters that I need as an output: ...
4answers
279 views

### Clustering algorithm which does not require to tell the number of clusters

I have a dataframe with 2 columns of numerical values. I want to apply a clustering algorithm to put all the entries into the same group, which have a relatively small distance to the other entries. ...
2answers
2k views

### Similarity Measure between two feature vectors

I have face identification system with following details: VGG16 model for feature extraction 512 dimensional feature vector (...
1answer
171 views

### Is the distance in Nearest Neighbors a good measure of similarity?

Let's train a Nearest Neighbor model with just one sample in it: In [48]: nn = NearestNeighbors().fit([[0, 1, 0, 0]]) So this one sample has just one significant ...
1answer
57 views

### Is there a way to cluster words based on how similarly they sound?

I have a list of words for a fictional world I've made (don't judge lol). My ultimate goal is to generate more words that sound like them through a markov generator, but for now, I'm trying to build ...
3answers
2k views

### Mahalanobis distance between two clusters

I want to calculate the Mahalanobis distance between cluster $a$ and cluster $b$, each consisting from a set of multidimensional points. Assuming no correlation, calculating the distance between a ...
1answer
27 views

### Very basic question: what is an accepted term for “linear order distance”

In data science we have "Manhattan Distance" as a slang term for Level 1 Distance and "Euclidean Distance" as a slang term for Level 2 Distance. Is there an accepted term for linear distance in memory ...
1answer
61 views

### Statistically Robust Distance Measure/Metric for comparing more than two network data series

I have about 30 lists of unequal length (some of which are triplicates of the data), corresponding to metrics relating to nodes of different graphs. I want to compare their similarity using a distance ...
2answers
107 views

### Collection Of Variable Length Sequences and Descriptions: A Search Problem

I have a tough problem and need some advice: Suppose I have a collection of variable length sequences, many of which are unique -- imagine the moves to a chess game, eg d4 Nf6 c4 g6 Nc3 Bg7 ...
4answers
299 views

### Distance measure for ternary feature

I have a data set consisting of 100 features each of which are ternary: values of -1 if it exists in one category, 0 if it doesn't exist, and 1 if it exists in the second category. For example ...
3answers
776 views

### clustering 2-dimensional euclidean vectors - appropriate dissimilarity measure

I've got a set of approx. 50 000 2-dimensional euclidean vectors which are connected with 20 groups, i.e. each group has approx. 2500 2-dimensional euclidean vectors. My data includes endpoints ...
0answers
494 views

### What is behind “A. Grothendieck scheme theory” in Mondobrain?

Mondobrain proposes a "big data" technology with: a new generation of algorithms based on A. Grothendieck scheme theory (Field Medal) that extract knowledge and rules from data without any ...
3answers
463 views

### Are there algorithms for clustering objects with pairwise distances, without computing all pairwise distances?

I'm looking for a clustering algorithm that clusters objects, by using their pairwise distances, without needing to calculate all pairwise distances. Normally pairwise clustering is done like this: (...
1answer
789 views

### How to evaluate distance in k-means clusters?

I try to use k-means clusters (using SQLserver + R), and I was wondering how we could estimate a distance correctly. For instance, if we consider Euclidean distance form the center of the clusters, ...
2answers
1k views

### Nearest Neighbors on mixed data types in high dimensions

I would like to be able to use nearest neighbors to attempt to find the most similar samples to a subclass of samples (think treated vs untreated) in a dataset with continuous, categorical, and text ...
1answer
2k views

### Knn and euclidean distance

I'm studying the knn classification algorithm. Why can the euclidean distance be considered a nice measure of affinity between examples ? In one dimension (1 attribute) this seems correct, but if I ...
2answers
751 views

### Measuring distance preservation in dimensionality reduction

I am looking to compare the distance preserved during dimension reductions for several techniques. I have read some papers on similar topics here and here. For example, I would like to use the ...
3answers
97 views

### Match a two items from two different receipts

I have two different invoices or receipts. One is a Purchase order one is something like a receipt(acknowledgement). Suppose I have ordered(PO) Wine: White Wine Red Wine Rose Wine And I receive ...
1answer
30 views

### What are the Most Dissimilar MNIST Digits?

Using whatever definition of dissimilarity over sets that you'd like, what are the most dissimilar two digits in MNIST? I was thinking that a reasonable approach to answering the question would be to ...
1answer
439 views

### values passed to user-defined distance function by KNeighborsClassifier is wrong

I have a data-set in which all features are binary and the class of each data-point is also binary. I am trying to use KNearestClassifier with a user-defined distance function as follows: ...
1answer
470 views

### How can I measure the similarity between 2 IP addresses? Is there any code to re-use?

I need to measure the similarity of 2 IP addresses. I could not find any sample code in scala or other languages to find the distance between 2 IPs.
1answer
258 views

### How to build an encoder using a distance matrix

I have a similarity/distance matrix: a | b | c a 0 | 1 | 2 b 1 | 0 | 3 c 2 | 3 | 0 I want to build an encoder/model that learns an n-dimensional ...
2answers
5k views

### Multidimensional Dynamic Time Warping Implementation in Python - confirm?

I believe that I implemented MDTW in python here but I don't know if I did it correctly. The results seem intuitive. Can someone look at this code and tell me if you see anything wrong? A lot of the ...
3answers
159 views

### Memory-efficient metric calculation for ultra high dimensional data

I am preparing for clustering the data which can be only represent as a extremely sparse binary vectors. Each of the objects is represented by a large set the binary features ($10^3$ ~ $10^6$), each ...
1answer
67 views

### Positive semidefinite kernel matrix from Gower distance

I have a dataframe with continuous and categorical variables and I want to obtain a kernel matrix for classification. The kernel matrix must be symmetric and positive semidefinite, so that no ...
1answer
147 views

### Clustering time series based on monotonic similarity

Context I am involved in a task of clustering 1500 time series of 500 observations into a few number of clusters. The time series share all the same observed property at different spatial locations, ...