Questions tagged [distance]
For question regarding distance between distributions or variables, such as Euclidean distance between points in n-space.
2
votes
2answers
37 views
Meaning of axes in a clustering plot
If you have n time series of rainfall measurements every hour (x=time, y=amount of rain), and compute the distance matrix between each pair of time series based on Dynamic Time Warping, and then plot ...
1
vote
1answer
23 views
Minkowski distance with Missing Values
Im currently doing a subject for data science, and have the following point that im trying to understand.
We are looking to calculate distance in data sets where values may not be present. Now i ...
1
vote
0answers
30 views
Proximity distance between two strings
How can I calculate the proximity between two strings
Ex: I need to find all the names in unstructured text and also need to look if there is any given string (i.e 'title') nearby
Input:
'...
1
vote
1answer
17 views
Appropriate similarity measure that highlights symmetrical values of features
I am trying to find an appropriate distance measure that reflects the differences of the vectors seen in the image below:
The green vector is compared with the blue one and the orange one.
Most of ...
1
vote
2answers
58 views
clustering 2-dimensional euclidean vectors - appropriate dissimilarity measure
I've got a set of approx. 50 000 2-dimensional euclidean vectors which are connected with 20 groups, i.e. each group has approx. 2500 2-dimensional euclidean vectors. My data includes endpoints ...
4
votes
3answers
62 views
Distance between very large discrete probability distributions
I have 192 countries where each country has some value for 1 million attributes which sum up to 1 (a discrete probability distribution). For any one country most of the values for the attributes are 0....
0
votes
0answers
27 views
Estimate the number of very distinct elements inside a dataset
Lets take for example the MNIST dataset, this dataset is composed of 60 000 training samples each sample is a handwritten digit between 0 and 9. however, we may find several samples inside this ...
1
vote
0answers
21 views
advice on distance metric for knn w/image recognition
I'm getting my feet wet with machine learning and am implementing a knn algorithm on a dataset that i've created. I've created a set of images of circles and squares and want the knn algorithm to ...
1
vote
1answer
20 views
Euclidean distance for more than 2 datapoints
I need to compare $n$ 3-dimensional vectors with $k$ 3-dimensional vectors using euclidean distance. Is that possible?
1
vote
1answer
19 views
Euclidean distance for more than two datapoints
I have n 3-dimensional vectors. Is there a way to find distance between all using euclidean distance?
0
votes
1answer
18 views
Voting patterns similarities
I'm interested in any research materials on voting patterns.
I have a data set of how PMs (members of parliament) voted in my country during last couple of years. Each PM has 3 buttons: Yes, No, ...
1
vote
1answer
223 views
How to calculate the silhouette coefficient?
Calculate the silhouette coefficient of point Pi from the above image.
To apply the given formula, how to know which is a(i) and b(i)?
2
votes
1answer
175 views
Knn and euclidean distance
I'm studying the knn classification algorithm.
Why can the euclidean distance be considered a nice measure of affinity between examples ?
In one dimension (1 attribute) this seems correct, but if I ...
1
vote
0answers
18 views
Suitability of Jaccard's distance for count vectorized features
I had an assignment in which we had to classify the cuisine and also give back the top-5 recipes based on given input. I did a count vectorization (countVectorize.transformer()) for the following data ...
2
votes
1answer
72 views
How can I measure the similarity between 2 IP addresses? Is there any code to re-use?
I need to measure the similarity of 2 IP addresses. I could not find any sample code in scala or other languages to find the distance between 2 IPs.
0
votes
0answers
30 views
How to better visualizing Earth Mover Distance in relation with generative adversarial networks?
After reading the original EMD paper from 1998 I am having a hard time trying to visualize the connections between their dirt pile example to generative adversarial networks. Everything is kind of ...
4
votes
2answers
361 views
Alternative distance to Dynamic Time Warping
I am performing a comparison among time series by using Dynamic Time Warping (DTW). However, it is not a real distance, but a distance-like quantity, since it doesn't assure the triangle inequality to ...
0
votes
0answers
12 views
How can I iterate over lists of data to calculate KS statistic for each pair of lists and then output a distance matrix?
I am looking to use this the Kolmogorov-Smirnov test function from Scipy, but I have a long list of pairs of lists to calculate the KS test for (and the data lists are of different length which makes ...
3
votes
1answer
45 views
Statistically Robust Distance Measure/Metric for comparing more than two network data series
I have about 30 lists of unequal length (some of which are triplicates of the data), corresponding to metrics relating to nodes of different graphs. I want to compare their similarity using a distance ...
3
votes
2answers
133 views
How the squared Euclidean distance is an example of non-metric function?
I am reading a book on Pattern Recognition (by Prof V Susheela Devi and Prof Murty) where in the chapter of data representation 2.3.3 the non metric similarity function is defined as those which do ...
2
votes
1answer
76 views
How to build an encoder using a distance matrix
I have a similarity/distance matrix:
a | b | c
a 0 | 1 | 2
b 1 | 0 | 3
c 2 | 3 | 0
I want to build an encoder/model that learns an n-dimensional ...
3
votes
2answers
63 views
Collection Of Variable Length Sequences and Descriptions: A Search Problem
I have a tough problem and need some advice:
Suppose I have a collection of variable length sequences, many of which are unique -- imagine the moves to a chess game, eg
d4 Nf6
c4 g6
Nc3 Bg7
...
0
votes
0answers
62 views
Why accuracy score drops in KNN if I change the wights to distance from uniform
I ran KNN in sklearn and the accuracy score dropped a bit when I changed the weights option to 'distance' from the default 'uniform'. Why is this? Is it not natural to have the importance of each ...
0
votes
1answer
221 views
What are similarity and distance metrics in classification?
I have an assignment to train a model to classify text data, the brief for the assignment mentions that any for any learning model used I have to provide a reasoning for the similarity or distance ...
0
votes
1answer
73 views
What is the efficient way to generate a similarity score when comparing two face images?
I am working on a face recognition application using deep learning. To plot the ROC curves and do performance evaluation, I extracted the features from the last layer of the deep neural network and I ...
1
vote
0answers
60 views
How tangent distance works in three-dimentional space
I've read about tangent distance used in machine learning but I don't really know how it works. Concretely, consider two very different manifolds M1 and M2 and two points x1, x2 on that two manifolds ...
0
votes
2answers
850 views
Finding similarity between two histogram plots
I have data census(name, sex, age, capital_gain) and I want to plot all possible views (in a histogram) and find the most interesting view based on the distinction to other views.
So, I have to ...
3
votes
1answer
340 views
Can I use euclidean distance for Latent Dirichlet Allocation document similarity?
I have a Latent Dirichlet Allocation (LDA) model with $K$ topics trained on a corpus with $M$ documents. Due to my hyper parameter configurations, the output topic distributions for each document is ...
-1
votes
2answers
147 views
Assessing Group Similarities and Dissimilarities Post PCA
The goal is to assess similarity and dissimilarity between 6 known groups.
The original data began with the 6 known groups and 2,700+ variables all on a scale of 0 to 100.
I have performed PCA to ...
2
votes
0answers
50 views
Outlier detection: Should the metric used in kNN take into account variance explained by each coordinate?
After applying PCA and working with the reduced dataset, I want to delete the outliers. To do this my idea is to compute the kNN-graph and delete those vertices (points) that have an inner degree of 0....
1
vote
1answer
172 views
Limits of Hellinger distance values
I am calculating Hellinger distance for different vectors. I initially assumed that the value returned by it in in the range of 0 to 1.
However for the following two vectors I received Hellinger ...
17
votes
1answer
2k views
What is Hellinger Distance and when to use it?
I am interested in knowing what really happens in Hellinger Distance (in simple terms). Furthermore, I am also interested in knowing what are types of problems that we can use Hellinger Distance? ...
-1
votes
1answer
47 views
Distance measure calculation addresses for record linking
At the moment we use different methods for record linking locations in different datasets.
Theoretically given two locations we can give a prediction on how well they match (are the same). This is ...
1
vote
1answer
77 views
how to compare two groups
I need to compare two groups of people, where the independent variable is having a college degree, and the dependent variable is the income.
The problem is that if I divide the whole population of ...
1
vote
0answers
15 views
DTW - What is fan-in nodes and why not just take the nearst points from each timeserie?
It has been a while that I am trying to understand the dtw (dynamic time warping) algorithm but I steel can't figure out the loginc behind this algorithm..
In many references it's explained that to ...
1
vote
0answers
327 views
2D Image-based Avatar Animation With Expressions / Image-To-Image Translation
What I want to do
Input an image-sequence (videoframes) that is already cropped and aligned to only the face/hair region and photorealistically replace this Person, Person A (can be anyone, I also ...
8
votes
3answers
6k views
When would one use Manhattan distance as opposite to Euclidean distance?
I am trying to look for a good argument on why one would use the Manhattan distance over the Euclidian distance in Machine Learning.
The closest thing I found to a good argument so far is on this MIT ...
0
votes
0answers
177 views
Finding unique set of nearest neighbors from symmetric distance matrix
I have computed nXn distance matrix starting from dataframe of order n. I have also computed nearest neighbors for datapoint based on the distance matrix with K=10. I'm using k.nearest.neighbor ...
1
vote
2answers
2k views
Multidimensional Dynamic Time Warping Implementation in Python - confirm?
I believe that I implemented MDTW in python here but I don't know if I did it correctly. The results seem intuitive. Can someone look at this code and tell me if you see anything wrong?
A lot of the ...
1
vote
1answer
320 views
How to compare performance of Cosine Similarity and Manhatten Distance?
I'm doing clustering of documents by applying k-Means on the word-vectors. To measure the cluster quality, I calculate David Bouldin Index for different k's. I tried two different distance measures, ...
4
votes
4answers
2k views
Is there a way to measure correlation between two similar datasets?
Let's say that I have two similar datasets with the same size of elements, for example 3D points :
Dataset A : { (1,2,3), (2,3,4), (4,2,1) }
Dataset B : { (2,1,3), (2,4,6), (8,2,3) }
And the ...
3
votes
4answers
171 views
Distance measure for ternary feature
I have a data set consisting of 100 features each of which are ternary: values of -1 if it exists in one category, 0 if it doesn't exist, and 1 if it exists in the second category. For example
...
1
vote
3answers
91 views
Memory-efficient metric calculation for ultra high dimensional data
I am preparing for clustering the data which can be only represent as a extremely sparse binary vectors. Each of the objects is represented by a large set the binary features (...
1
vote
1answer
68 views
How to test People similarity measure?
I am doing a project on finding famous people who are similar to each other.
For this, I am extracting a bunch of features and applying a distance function on them to evaluate who is closer to whom.
...
0
votes
1answer
470 views
Dimension reduction techniques in R that do not use the full distance matrix
I try to apply non-linear dimension reduction in R. As usual in machine learning I have a large data set (100 K rows). I tried the packages
RDRToolbox and ...
1
vote
3answers
122 views
Looking for an algorithm that correctly clusters visually separable clusters
I have visualized a dataset in 2D after employing PCA. As 2D visualization shows in figure, there is a good separation between points (A, B). Now, I want to use a metric which can separate these ...
1
vote
1answer
1k views
How do I find the minimum distance between zip codes in R?
I have a dataset that lists all zip codes in the U.S., their types (standard, po box, university, etc). I want to replace po box and university zip codes with the next closest standard zip code. I ...
2
votes
0answers
392 views
What is behind “A. Grothendieck scheme theory” in Mondobrain?
Mondobrain proposes a "big data" technology with:
a new generation of algorithms based on A. Grothendieck scheme theory
(Field Medal) that extract knowledge and rules from data without any
...
3
votes
1answer
335 views
How to evaluate distance in k-means clusters?
I try to use k-means clusters (using SQLserver + R), and I was wondering how we could estimate a distance correctly.
For instance, if we consider Euclidean distance form the center of the clusters, ...
3
votes
3answers
666 views
Mahalanobis distance between two clusters
I want to calculate the Mahalanobis distance between cluster $a$ and cluster $b$, each consisting from a set of multidimensional points. Assuming no correlation, calculating the distance between a ...
