Questions tagged [distance]

For question regarding distance between distributions or variables, such as Euclidean distance between points in n-space.

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

How to calculate Fuzzy C-Means problem by hand

I figured that this doubt next can interest another students like me and help others also that are trying to understand mathematically the fuzzy c-means mathematical mechanism already that some books ...
1
vote
1answer
24 views

It seems that the output of sklearn.metrics.pairwise.euclidean_distances is different to the formula on doc

The doc of sklearn.metrics.pairwise.euclidean_distances() gives this formula dist(x, y) = sqrt(dot(x, x) - 2 * dot(x, y) + dot(y, y)). Apply this formula to ...
0
votes
1answer
55 views

Implementing Mahalanobis Distance in Python

I am trying to implementing Mahalanobis Distance from scratch but I am getting an error- The formula for Mahalanobis Distance is- Now my code is- ...
0
votes
0answers
10 views

What options do I have for measuring similarities by using vectors generated from texts?

I have a data set which contains vectors generated from subtitles, I want to measure the similarity between each pair of the observation. Now I have tried L1, L2, cosine similarity and Mahalanobis ...
0
votes
0answers
16 views

Is it necessary to perform Z-score or Min-Max normalization on L1-normalized data?

I have a dataset which contains vectors that generated from subtitles and have been L1 normalised, I want to calculate cosine similarity & Euclidean distance, I thought it is better if I use Z-...
2
votes
1answer
43 views

values passed to user-defined distance function by KNeighborsClassifier is wrong

I have a data-set in which all features are binary and the class of each data-point is also binary. I am trying to use KNearestClassifier with a user-defined distance function as follows: ...
0
votes
0answers
18 views

CNN Architecture for Measuring Object Distances

I'm trying to use CNNs for infering object distances from an image. The input images correspond to states of a 2D game: Game states are not represented as images but as matrixes of observations. ...
0
votes
0answers
17 views

Is there any paper introduce an intuitive method for clustering evaluation?

I would like to use the most intuitive method like minimizing the within-cluster distance and maximizing the distance between neighbouring clusters, but not sure does this method have a name or ...
3
votes
1answer
30 views

Is the distance in Nearest Neighbors a good measure of similarity?

Let's train a Nearest Neighbor model with just one sample in it: In [48]: nn = NearestNeighbors().fit([[0, 1, 0, 0]]) So this one sample has just one significant ...
6
votes
0answers
208 views

Finding linear transformation under which distance matrices are similar

I have n sets of vectors, where each set S_i contains k vectors in ...
0
votes
0answers
43 views

Cluster based on both positions and similarity scores

I have a dataframe position giving me the x and y positions of 87 points. I also have a 87 x 87 similarity matrix giving me the pairwise similarity scores between ...
0
votes
2answers
25 views

Estimating time to travel between two lat/longs

I'm trying to create an offline estimator for how long it would take to get from one lat/long to another. Two approaches I have come across are the Haversine distance and the Manhattan distance. What ...
0
votes
1answer
20 views

learning a distance metric from the output of a knn output

Suppose that someone has trained a nearest-neighbor algorithm based on some unknown metric. I have a large dataset of $N$ observations, and $P$ features. For each observation I am given $K$ indices ...
0
votes
0answers
22 views

Features Vectors in embedding space

I have a bunch of users, each of them with about 100 features. My goal is to create an embedded space to compute the "distance" between users. Also, I want to be able to visualize it with Tensorboard (...
4
votes
6answers
164 views

Distance between users

I want to compute the "distance" between users in order to return the top n similar users, for any given user. For each user a have a bunch of features. This is close to a recommendation system, ...
2
votes
1answer
47 views

Coordinate System's influence on $L$ distances (Manhattan and Euclidean)

I don't understand this picture, which says if we change the coordinate system, we would have the same result for $L_2$ distance, whereas, our result would differ for $L_1$ distance. What does it ...
0
votes
0answers
97 views

Implementation of Bhattacharyya distance for filtering images that are far off from their cluster

I need assistance with the python implementation of Bhattacharyya-distance for the below use case: Here, the polygons P1, P2..Pn refer to the different images where each pixel is represented by 'n' ...
1
vote
0answers
26 views

What is divergence exactly in machine learning?

I know about KL divergence, JS Divergence and clearly know that it is different from the divergence in calculus. I have an intutive feeling of divergence as it roughly compares the closeness of two ...
2
votes
1answer
151 views

Measuring distance preservation in dimensionality reduction

I am looking to compare the distance preserved during dimension reductions for several techniques. I have read some papers on similar topics here and here. For example, I would like to use the ...
3
votes
1answer
71 views

K Nearest Neighbour with different distance matrix to each datapoint

I'm wondering if there is library support in python (such as sklearn) for doing KNN on a data set that has a custom distance matrix (positive definite) for each data point (x is a query point, $x_i$ ...
1
vote
1answer
264 views

What does sklearn's pairwise_distances with metric='correlation' do?

I've put different values into this function and observed the output. But I can't find a predictable pattern in what is being outputed. Then I tried digging through the function itself, but its ...
0
votes
1answer
51 views

scipy.spatial.distance.mahalanobis return null values for some vectors in python

I am using scipy.spatial.distance.mahalanobis to calculate distance between two vectors but i'm getting null values for some vector I don't know why? null value is possible?
1
vote
1answer
157 views

How to measaure the similarity between two series?

I'm confused about how to measure the similarity between two time series with the same length. For example, both time series are 2 hours in length and every 5 minutes a point. I really want to know ...
0
votes
1answer
26 views

Date transformation for KNN to get distance [duplicate]

I have data set with date features like 01/01/2019 and I would like to use KNN. However, I cannot find a good transformation for dates that has a meaningful distance result for the last feature. For ...
0
votes
2answers
64 views

Date transformation for KNN

I have data set with date features like 01/01/2019 and I would like to use KNN. However, I cannot find a good transformation for dates that has a meaningful ...
2
votes
1answer
29 views

Positive semidefinite kernel matrix from Gower distance

I have a dataframe with continuous and categorical variables and I want to obtain a kernel matrix for classification. The kernel matrix must be symmetric and positive semidefinite, so that no ...
2
votes
3answers
64 views

Are there algorithms for clustering objects with pairwise distances, without computing all pairwise distances?

I'm looking for a clustering algorithm that clusters objects, by using their pairwise distances, without needing to calculate all pairwise distances. Normally pairwise clustering is done like this: (...
2
votes
3answers
513 views

Clustering algorithm for a distance matrix

I have a similarity matrix between N objects. For each N objects, I have a measure of how similar they are between each others - 0 being identical (the main diagonal) and increasing values as they get ...
2
votes
3answers
47 views

Match a two items from two different receipts

I have two different invoices or receipts. One is a Purchase order one is something like a receipt(acknowledgement). Suppose I have ordered(PO) Wine: White Wine Red Wine Rose Wine And I receive ...
1
vote
0answers
17 views

Which distance method will account for positions of observations in arrays?

I am trying to figure out the distance method suited for my problem. I have different arrays of the same length. Each position in the array matters. Let's say ... each array is representing an image, ...
2
votes
1answer
20 views

Is there a way to cluster words based on how similarly they sound?

I have a list of words for a fictional world I've made (don't judge lol). My ultimate goal is to generate more words that sound like them through a markov generator, but for now, I'm trying to build ...
2
votes
1answer
68 views

Clustering time series based on monotonic similarity

Context I am involved in a task of clustering 1500 time series of 500 observations into a few number of clusters. The time series share all the same observed property at different spatial locations, ...
2
votes
3answers
47 views

A metric between trees

I have certain tree structures. I am not an expert in machine learning. As I would with take KNN, I would calculate distances via metric function and a new data point and the points from the training ...
1
vote
1answer
294 views

Why don't we use Manhattan distance instead of euclidean distance in linear regression?

When I am explaining concept of linear regression to one of my peers, I got stuck in answer this question. Why don`t we use Manhattan distance instead of euclidean distance in linear regression? Can ...
0
votes
1answer
71 views

How can i use Hellinger Distance on array of different length?

I have to use Hellinger distance to compare arrays that are not the same length. How do you do this correctly? Putting a zero in the missing fields for the shorter array does not sound like the best ...
1
vote
2answers
317 views

How can I implement tangent distance for k-nearest neighbor in python/scikit-learn?

My ultimate aim is to have a function which I can feed into scikit-learn's NearestNeighbor class as a custom metric parameter. ...
0
votes
2answers
273 views

Nearest Neighbors on mixed data types in high dimensions

I would like to be able to use nearest neighbors to attempt to find the most similar samples to a subclass of samples (think treated vs untreated) in a dataset with continuous, categorical, and text ...
1
vote
0answers
9 views

How to generate multiple vectors given their distances [closed]

Well, the title says it all. I'm looking for an algorithm that would generate multiple random vectors with given distances between them. Any suggestions?
1
vote
1answer
211 views

Calculation of distance between samples in data mining

I am confused about a little issue related to distance calculation. What I want to know is, while calculating the distance between samples in classification or regression, is the label or output class ...
2
votes
2answers
371 views

Meaning of axes in a clustering plot

If you have n time series of rainfall measurements every hour (x=time, y=amount of rain), and compute the distance matrix between each pair of time series based on Dynamic Time Warping, and then plot ...
1
vote
1answer
106 views

Minkowski distance with Missing Values

Im currently doing a subject for data science, and have the following point that im trying to understand. We are looking to calculate distance in data sets where values may not be present. Now i ...
1
vote
0answers
57 views

Proximity distance between two strings

How can I calculate the proximity between two strings Ex: I need to find all the names in unstructured text and also need to look if there is any given string (i.e 'title') nearby Input: '...
1
vote
1answer
27 views

Appropriate similarity measure that highlights symmetrical values of features

I am trying to find an appropriate distance measure that reflects the differences of the vectors seen in the image below: The green vector is compared with the blue one and the orange one. Most of ...
2
votes
3answers
244 views

clustering 2-dimensional euclidean vectors - appropriate dissimilarity measure

I've got a set of approx. 50 000 2-dimensional euclidean vectors which are connected with 20 groups, i.e. each group has approx. 2500 2-dimensional euclidean vectors. My data includes endpoints ...
4
votes
3answers
72 views

Distance between very large discrete probability distributions

I have 192 countries where each country has some value for 1 million attributes which sum up to 1 (a discrete probability distribution). For any one country most of the values for the attributes are 0....
1
vote
0answers
86 views

advice on distance metric for knn w/image recognition

I'm getting my feet wet with machine learning and am implementing a knn algorithm on a dataset that i've created. I've created a set of images of circles and squares and want the knn algorithm to ...
1
vote
1answer
67 views

Euclidean distance for more than 2 datapoints

I need to compare $n$ 3-dimensional vectors with $k$ 3-dimensional vectors using euclidean distance. Is that possible?
1
vote
1answer
38 views

Euclidean distance for more than two datapoints

I have n 3-dimensional vectors. Is there a way to find distance between all using euclidean distance?
1
vote
1answer
22 views

Voting patterns similarities

I'm interested in any research materials on voting patterns. I have a data set of how PMs (members of parliament) voted in my country during last couple of years. Each PM has 3 buttons: Yes, No, ...
3
votes
2answers
4k views

How to calculate the silhouette coefficient?

Calculate the silhouette coefficient of point Pi from the above image. To apply the given formula, how to know which is a(i) and b(i)?