Questions tagged [distance]
For question regarding distance between distributions or variables, such as Euclidean distance between points in n-space.
106
questions
0
votes
1answer
16 views
Clustering without information about identifier
I have a data-set with different products and binary value if it was sold in a store or not. I looks like:
...
6
votes
2answers
81 views
How do I test a difference between two proportions representing fatality rate for Covid 19 in Philippines and World (except Philippines)?
I'm trying to analyse if the fatality rate from my country (A third world country) vary significantly from the world's fatality rate.
So I'd basically have two samples, labeled (Philippines) and (...
0
votes
0answers
10 views
Learnable distance measure for segments of periodic multichannel signal?
We have a set of users, each with several sessions, each session represented as multichannel signal. Users behave differently, but if we visualize several sessions of the same user we can often see a ...
3
votes
4answers
272 views
Clustering algorithm which does not require to tell the number of clusters
I have a dataframe with 2 columns of numerical values. I want to apply a clustering algorithm to put all the entries into the same group, which have a relatively small distance to the other entries. ...
3
votes
1answer
24 views
Very basic question: what is an accepted term for “linear order distance”
In data science we have "Manhattan Distance" as a slang term for Level 1 Distance and "Euclidean Distance" as a slang term for Level 2 Distance. Is there an accepted term for linear distance in memory ...
4
votes
2answers
54 views
Cluster elements that appear in the same lists
Suppose I have a multitude of sets with (unordered) combinations of elements and I want to determine which elements tend to appear together.
For example
Given the following sets:
...
1
vote
0answers
22 views
How to have an insight of very similar processing methods by measuring the distance between observations produced by them? [closed]
Suppose that we use very similar methods(suppose now there are two such similar methods) to process upon each sample of a dataset. For each sample of the dataset with one of the two methods, we obtain ...
1
vote
1answer
27 views
Some robust perhaps more complex as neural networks to measure the distance between data? [closed]
Now after certain preprocess we can repeatedly get two sets of data points, exhaustively enumerate one from the first set and one from the second set to form pairs. Each pair of these two data points ...
2
votes
2answers
845 views
How to evaluate the K-Modes Clusters?
K-modes algorithm is available here
I want to do clustering of my binary dataset. I need to specify the number of clusters that I need as an output:
...
1
vote
0answers
42 views
Voronoi Diagram Algorithm for Geo Locations? [closed]
As per my Understanding:
Voronoi diagram: A Voronoi diagram is a simple concept, and it's based on the minimal distance needed to reach a landmark. If you need to go to a metro station, the most ...
0
votes
0answers
19 views
Random Forest proximity matrix with or without in bag samples
I'm running Random Forest classifications; and PCA plots based on the resulting proximity matrices are quite helpful to my analyses.
However, I found a comment on the R ...
0
votes
0answers
10 views
How to quickly find min, max and mean distance between samples fitted in Nearest Neighbors?
I am familiar only with SciKit Learn implementation of Nearest Neighbors model where one can fit data and execute K-neighbors or radius queries. This model does not provide any statistical information ...
0
votes
2answers
269 views
Question about Similarity vs Dissimilarity Matrix
Right now, I'm working on a coming up with a similarity vs dissimilarity matrix for a set of data points for a clustering algorithm. My question is, if I want to use one of the many clustering ...
0
votes
2answers
34 views
Question About Coming Up With Own Function for Distance Matrix (For Clustering)
Right now, I am currently working on implementing a clustering algorithm with millions data entries with regards to game users for a mobile game.
A lot of the features I plan on using are unique to ...
0
votes
0answers
13 views
Finding the closest neighbour of multidimensial data point
I have a test point with 15 attributes. I want to find the closest data point to this test point from 10,000 data points.
I'm thinking using something like this:
...
0
votes
0answers
51 views
How to calculate Fuzzy C-Means problem by hand
I figured that this doubt next can interest another students like me and help others also that are trying to understand mathematically the fuzzy c-means mathematical mechanism already that some books ...
1
vote
1answer
65 views
It seems that the output of sklearn.metrics.pairwise.euclidean_distances is different to the formula on doc
The doc of sklearn.metrics.pairwise.euclidean_distances() gives this formula
dist(x, y) = sqrt(dot(x, x) - 2 * dot(x, y) + dot(y, y)).
Apply this formula to ...
0
votes
0answers
12 views
What options do I have for measuring similarities by using vectors generated from texts?
I have a data set which contains vectors generated from subtitles, I want to measure the similarity between each pair of the observation. Now I have tried L1, L2, cosine similarity and Mahalanobis ...
0
votes
0answers
28 views
Is it necessary to perform Z-score or Min-Max normalization on L1-normalized data?
I have a dataset which contains vectors that generated from subtitles and have been L1 normalised, I want to calculate cosine similarity & Euclidean distance, I thought it is better if I use Z-...
2
votes
1answer
165 views
values passed to user-defined distance function by KNeighborsClassifier is wrong
I have a data-set in which all features are binary and the class of each data-point is also binary. I am trying to use KNearestClassifier with a user-defined distance function as follows:
...
0
votes
0answers
41 views
CNN Architecture for Measuring Object Distances
I'm trying to use CNNs for infering object distances from an image. The input images correspond to states of a 2D game:
Game states are not represented as images but as matrixes of observations. ...
0
votes
0answers
18 views
Is there any paper introduce an intuitive method for clustering evaluation?
I would like to use the most intuitive method like minimizing the within-cluster distance and maximizing the distance between neighbouring clusters, but not sure does this method have a name or ...
3
votes
1answer
58 views
Is the distance in Nearest Neighbors a good measure of similarity?
Let's train a Nearest Neighbor model with just one sample in it:
In [48]: nn = NearestNeighbors().fit([[0, 1, 0, 0]])
So this one sample has just one significant ...
8
votes
0answers
258 views
Finding linear transformation under which distance matrices are similar
I have n sets of vectors, where each set S_i contains k vectors in ...
0
votes
0answers
92 views
Cluster based on both positions and similarity scores
I have a dataframe position giving me the x and y positions of 87 points.
I also have a 87 x 87 similarity matrix giving me the pairwise similarity scores between ...
0
votes
4answers
63 views
Estimating time to travel between two lat/longs
I'm trying to create an offline estimator for how long it would take to get from one lat/long to another. Two approaches I have come across are the Haversine distance and the Manhattan distance. What ...
1
vote
2answers
47 views
Reverse engineering a distance metric from the output of a k-NN
Suppose that someone has trained a nearest-neighbor algorithm based on some unknown metric. I have a large dataset of $N$ observations and $P$ features. For each observation, I am given $K$ indices ...
0
votes
0answers
31 views
Features Vectors in embedding space
I have a bunch of users, each of them with about 100 features.
My goal is to create an embedded space to compute the "distance" between users.
Also, I want to be able to visualize it with Tensorboard (...
4
votes
6answers
218 views
Distance between users
I want to compute the "distance" between users in order to return the top n similar users, for any given user.
For each user a have a bunch of features.
This is close to a recommendation system, ...
2
votes
1answer
149 views
Coordinate System's influence on $L$ distances (Manhattan and Euclidean)
I don't understand this picture, which says if we change the coordinate system, we would have the same result for $L_2$ distance, whereas, our result would differ for $L_1$ distance.
What does it ...
0
votes
0answers
276 views
Implementation of Bhattacharyya distance for filtering images that are far off from their cluster
I need assistance with the python implementation of Bhattacharyya-distance for the below use case:
Here, the polygons P1, P2..Pn refer to the different images where each pixel is represented by 'n' ...
1
vote
0answers
54 views
What is divergence exactly in machine learning?
I know about KL divergence, JS Divergence and clearly know that it is different from the divergence in calculus. I have an intutive feeling of divergence as it roughly compares the closeness of two ...
2
votes
2answers
358 views
Measuring distance preservation in dimensionality reduction
I am looking to compare the distance preserved during dimension reductions for several techniques. I have read some papers on similar topics here and here.
For example, I would like to use the ...
3
votes
1answer
416 views
K Nearest Neighbour with different distance matrix to each datapoint
I'm wondering if there is library support in python (such as sklearn) for doing KNN on a data set that has a custom distance matrix (positive definite) for each data point (x is a query point, $x_i$ ...
1
vote
1answer
860 views
What does sklearn's pairwise_distances with metric='correlation' do?
I've put different values into this function and observed the output. But I can't find a predictable pattern in what is being outputed.
Then I tried digging through the function itself, but its ...
0
votes
1answer
149 views
scipy.spatial.distance.mahalanobis return null values for some vectors in python
I am using scipy.spatial.distance.mahalanobis to calculate distance between two vectors but i'm getting null values for some vector I don't know why? null value is possible?
1
vote
1answer
814 views
How to measaure the similarity between two series?
I'm confused about how to measure the similarity between two time series with the same length. For example, both time series are 2 hours in length and every 5 minutes a point. I really want to know ...
0
votes
1answer
30 views
Date transformation for KNN to get distance [duplicate]
I have data set with date features like 01/01/2019 and I would like to use KNN. However, I cannot find a good transformation for dates that has a meaningful distance result for the last feature.
For ...
0
votes
2answers
94 views
Date transformation for KNN
I have data set with date features like 01/01/2019 and I would like to use KNN. However, I cannot find a good transformation for dates that has a meaningful ...
2
votes
1answer
39 views
Positive semidefinite kernel matrix from Gower distance
I have a dataframe with continuous and categorical variables and I want to obtain a kernel matrix for classification. The kernel matrix must be symmetric and positive semidefinite, so that no ...
2
votes
3answers
109 views
Are there algorithms for clustering objects with pairwise distances, without computing all pairwise distances?
I'm looking for a clustering algorithm that clusters objects, by using their pairwise distances, without needing to calculate all pairwise distances.
Normally pairwise clustering is done like this: (...
2
votes
3answers
1k views
Clustering algorithm for a distance matrix
I have a similarity matrix between N objects. For each N objects, I have a measure of how similar they are between each others - 0 being identical (the main diagonal) and increasing values as they get ...
2
votes
3answers
54 views
Match a two items from two different receipts
I have two different invoices or receipts. One is a Purchase order one is something like a receipt(acknowledgement).
Suppose I have ordered(PO) Wine:
White Wine
Red Wine
Rose Wine
And I receive ...
1
vote
0answers
17 views
Which distance method will account for positions of observations in arrays?
I am trying to figure out the distance method suited for my problem. I have different arrays of the same length. Each position in the array matters.
Let's say ...
each array is representing an image, ...
2
votes
1answer
23 views
Is there a way to cluster words based on how similarly they sound?
I have a list of words for a fictional world I've made (don't judge lol).
My ultimate goal is to generate more words that sound like them through a markov generator, but for now, I'm trying to build ...
2
votes
1answer
98 views
Clustering time series based on monotonic similarity
Context
I am involved in a task of clustering 1500 time series of 500 observations into a few number of clusters. The time series share all the same observed property at different spatial locations, ...
4
votes
3answers
69 views
A metric between trees
I have certain tree structures. I am not an expert in machine learning.
As I would with take KNN, I would calculate distances via metric function and a new data point and the points from the training ...
1
vote
1answer
571 views
Why don't we use Manhattan distance instead of euclidean distance in linear regression?
When I am explaining concept of linear regression to one of my peers, I got stuck in answer this question. Why don`t we use Manhattan distance instead of euclidean distance in linear regression? Can ...
0
votes
1answer
114 views
How can i use Hellinger Distance on array of different length?
I have to use Hellinger distance to compare arrays that are not the same length.
How do you do this correctly? Putting a zero in the missing fields for the shorter array does not sound like the best ...
1
vote
2answers
445 views
How can I implement tangent distance for k-nearest neighbor in python/scikit-learn?
My ultimate aim is to have a function which I can feed into scikit-learn's NearestNeighbor class as a custom metric parameter.
...
