Questions tagged [distance]
For question regarding distance between distributions or variables, such as Euclidean distance between points in n-space.
123
questions
1
vote
1answer
16 views
How to calculate the distance between two locations using Haversine Formula? [closed]
I have the columns of Latitude and Longitude of city like shown below :
...
0
votes
1answer
10 views
Treat similar observations in a classification problem
I have a dataset of about 200k rows and I'm working on a classification problem.
Grouping the dataset by a key variable, I noticed that some rows, with the same key value, have similar values in other ...
0
votes
0answers
14 views
Can a siamese neural network distinguish expected from unexpected changes?
Please redirect me to another stack exchange if this isn't the appropriate forum. I am interested in finding a neural network architecture that can detect distance between two inputs. As an example, ...
0
votes
1answer
19 views
Non-commutative distance formula
I am trying to find a distance formula or a method that can give the non-commutative distance between two points in a feature space.
Suppose there are two movies represented in an R^n feature space. ...
0
votes
0answers
12 views
Can I use depth prediction map to infer horizontal distances?
I have a hardware + software setup that uses a sensor to give good estimates of depth, onto a pixel map - think Kinect or similar. Example below for context:
Now assume I can access individual pixel ...
0
votes
0answers
30 views
2D Z-score/Mahalanobis distance that includes a penalty for uncertainty
I have some 2D points and I want to assess their performance against the target point.
When I was doing this in 1D, I took the Z-score Z = (x- mu)/sigma, but that ...
1
vote
0answers
19 views
Compare distance between embeddings in different dimensions
I am working on a problem with CNNs.
After the convolutional layers, comes a "flatten". One could interpret that as a representation of the input image in some high-dimensional continuous ...
1
vote
1answer
52 views
Why does an imbalanced data set badly effect distance measures like Mahalanobis?
I'm relatively new to data science and I am struggling to understand why the Mahalanobis distance (or any other distance measure) applied to an imbalanced data-set becomes inaccurate. I have a data ...
1
vote
1answer
47 views
How to estimate real distance between two detected objects in an image?
You may think this is a duplicate, but my situation is different than previously asked questions. The only information I have is the width and height of the bounding boxes of detected people. The ...
0
votes
1answer
21 views
Siamese vs matching network for correct image category matching
I have to find the closest match between my image and bunch of already collected images of different classes in the folder. Whic meta-learning approach should I select. I am thinking about the Siamese ...
0
votes
0answers
14 views
What are practical benefits of the various distance metrics in scipy?
I’m looking for a information about distance metrics.
Python’s scipy gives many different metrics, and I’d like to know more about their practical use.
In particular, I’m trying to understand the ...
0
votes
0answers
38 views
Use Machine Learning/Neural Network + Distance Measurements to Find the Position of Devices (Localization)
I want to find the position of several devices using at least distance measurements. These measurements are done using a radio, and it might be that not all devices are in radio range (no distance ...
0
votes
0answers
9 views
Distance between two time-dependent distribution
In my research, I want to use a meaningful and computationally tractable distance between two time-dependent probability distributions. For stationary distributions, several distance measures are used,...
0
votes
1answer
26 views
Correlation/distance between sparse vectors
I am looking for a metric for comparing gene count tables. These are long columns of data (a few millions genes by a few dozen samples), with all non-negative entries, about 90% of which are zeros. ...
0
votes
0answers
19 views
What's the best way to detect crowds?
I have a dictionary containing people and the distance between each pair in the following format:
...
2
votes
1answer
28 views
What are the Most Dissimilar MNIST Digits?
Using whatever definition of dissimilarity over sets that you'd like, what are the most dissimilar two digits in MNIST? I was thinking that a reasonable approach to answering the question would be to ...
0
votes
0answers
28 views
Proximity multi-dimentional arrays. Which algorithms are commonly used?
I'm new to data science so still learning so much. I've been searching for proximity algorithms but I'm not sure which are suited for multi-dimensional arrays. Any guidance would be greatly ...
3
votes
1answer
663 views
Why is the cosine distance used to measure the similatiry between word embeddings?
While computing the similarity between the words, cosine similarity or distance is computed on word vectors. Why aren't other distance metrics such as Euclidean ...
0
votes
0answers
27 views
best distance measure for linked data
hello let suppose that I have an ndarray for linked data which look like this
...
4
votes
1answer
75 views
KNN Regression: Distance function and/or vector representation for datetime features
Context: Trying to forecast some sort of consumption value (e.g. water) using datetime features and exogenous variables (like temperature).
Take some datetime features like week days (...
0
votes
0answers
40 views
Why do all DTW-packages use different step-patterns and distance-methods?
I already posted it on StackOverflow. I was not sure if this was the right place.
I have several questions about the algorithm DTW. I tried different Python-Packages
DTAI-Distance
DTW-Python
First ...
3
votes
2answers
1k views
Similarity Measure between two feature vectors
I have face identification system with following details:
VGG16 model for feature extraction
512 dimensional feature vector (...
0
votes
0answers
28 views
Bottleneck Distance
Is there a range of values for the bottleneck distance in persim package (python) to conclude that the two datasets are similar?
Also, does it make sense to compute the bottleneck distance using $H_0$ ...
1
vote
0answers
27 views
Pairwise 3D object correlation between 2 objects
I have a dataset which contains 3D CT scans from different patients along with the segmenation masks of a certain organ. The 3D scans have been drawn each day for a period of 30 days for each patient. ...
0
votes
0answers
15 views
Calculate distance ignoring Outliers
I have the GPS Coordinates of a person performing an activity. The activity has a start GPS point, start time and end GPS point, end time.
The rule is that the difference between the start GPS point ...
0
votes
1answer
11 views
How to measure the distance (in generalized sense) between geographical regions? [closed]
I need to construct a distance matrix for a few U.S. counties that are adjacent to one or another, and choosing the definition of distance is very tricky. The shortest path (i.e the minimum number of ...
0
votes
1answer
30 views
Clustering without information about identifier
I have a data-set with different products and binary value if it was sold in a store or not. I looks like:
...
6
votes
2answers
103 views
How do I test a difference between two proportions representing fatality rate for Covid 19 in Philippines and World (except Philippines)?
I'm trying to analyse if the fatality rate from my country (A third world country) vary significantly from the world's fatality rate.
So I'd basically have two samples, labeled (Philippines) and (...
3
votes
4answers
278 views
Clustering algorithm which does not require to tell the number of clusters
I have a dataframe with 2 columns of numerical values. I want to apply a clustering algorithm to put all the entries into the same group, which have a relatively small distance to the other entries. ...
3
votes
1answer
27 views
Very basic question: what is an accepted term for “linear order distance”
In data science we have "Manhattan Distance" as a slang term for Level 1 Distance and "Euclidean Distance" as a slang term for Level 2 Distance. Is there an accepted term for linear distance in memory ...
5
votes
3answers
240 views
Cluster elements that appear in the same lists
Suppose I have a multitude of sets with (unordered) combinations of elements and I want to determine which elements tend to appear together.
For example
Given the following sets:
...
1
vote
0answers
23 views
How to have an insight of very similar processing methods by measuring the distance between observations produced by them? [closed]
Suppose that we use very similar methods(suppose now there are two such similar methods) to process upon each sample of a dataset. For each sample of the dataset with one of the two methods, we obtain ...
1
vote
1answer
27 views
Some robust perhaps more complex as neural networks to measure the distance between data? [closed]
Now after certain preprocess we can repeatedly get two sets of data points, exhaustively enumerate one from the first set and one from the second set to form pairs. Each pair of these two data points ...
3
votes
2answers
5k views
How to evaluate the K-Modes Clusters?
K-modes algorithm is available here
I want to do clustering of my binary dataset. I need to specify the number of clusters that I need as an output:
...
1
vote
0answers
99 views
Voronoi Diagram Algorithm for Geo Locations? [closed]
As per my Understanding:
Voronoi diagram: A Voronoi diagram is a simple concept, and it's based on the minimal distance needed to reach a landmark. If you need to go to a metro station, the most ...
0
votes
2answers
541 views
Question about Similarity vs Dissimilarity Matrix
Right now, I'm working on a coming up with a similarity vs dissimilarity matrix for a set of data points for a clustering algorithm. My question is, if I want to use one of the many clustering ...
0
votes
2answers
57 views
Question About Coming Up With Own Function for Distance Matrix (For Clustering)
Right now, I am currently working on implementing a clustering algorithm with millions data entries with regards to game users for a mobile game.
A lot of the features I plan on using are unique to ...
1
vote
1answer
186 views
It seems that the output of sklearn.metrics.pairwise.euclidean_distances is different to the formula on doc
The doc of sklearn.metrics.pairwise.euclidean_distances() gives this formula
dist(x, y) = sqrt(dot(x, x) - 2 * dot(x, y) + dot(y, y)).
Apply this formula to ...
2
votes
1answer
416 views
values passed to user-defined distance function by KNeighborsClassifier is wrong
I have a data-set in which all features are binary and the class of each data-point is also binary. I am trying to use KNearestClassifier with a user-defined distance function as follows:
...
0
votes
0answers
18 views
Is there any paper introduce an intuitive method for clustering evaluation?
I would like to use the most intuitive method like minimizing the within-cluster distance and maximizing the distance between neighbouring clusters, but not sure does this method have a name or ...
3
votes
1answer
151 views
Is the distance in Nearest Neighbors a good measure of similarity?
Let's train a Nearest Neighbor model with just one sample in it:
In [48]: nn = NearestNeighbors().fit([[0, 1, 0, 0]])
So this one sample has just one significant ...
13
votes
1answer
452 views
Finding linear transformation under which distance matrices are similar
I have $n$ sets of vectors, where each set $S_i$ contains $k$ vectors in $\mathbb{R}^d$. I know there is some unknown linear transformation $W$ under which the distance matrix $D_i$ (a $k\times k$ ...
0
votes
0answers
138 views
Cluster based on both positions and similarity scores
I have a dataframe position giving me the x and y positions of 87 points.
I also have a 87 x 87 similarity matrix giving me the pairwise similarity scores between ...
0
votes
4answers
199 views
Estimating time to travel between two lat/longs
I'm trying to create an offline estimator for how long it would take to get from one lat/long to another. Two approaches I have come across are the Haversine distance and the Manhattan distance. What ...
1
vote
2answers
58 views
Reverse engineering a distance metric from the output of a k-NN
Suppose that someone has trained a nearest-neighbor algorithm based on some unknown metric. I have a large dataset of $N$ observations and $P$ features. For each observation, I am given $K$ indices ...
0
votes
0answers
36 views
Features Vectors in embedding space
I have a bunch of users, each of them with about 100 features.
My goal is to create an embedded space to compute the "distance" between users.
Also, I want to be able to visualize it with Tensorboard (...
4
votes
6answers
370 views
Distance between users
I want to compute the "distance" between users in order to return the top n similar users, for any given user.
For each user a have a bunch of features.
This is close to a recommendation system, ...
5
votes
2answers
859 views
Coordinate System's influence on $L$ distances (Manhattan and Euclidean)
I don't understand this picture, which says if we change the coordinate system, we would have the same result for $L_2$ distance, whereas, our result would differ for $L_1$ distance.
What does it ...
1
vote
0answers
70 views
What is divergence exactly in machine learning?
I know about KL divergence, JS Divergence and clearly know that it is different from the divergence in calculus. I have an intutive feeling of divergence as it roughly compares the closeness of two ...
2
votes
2answers
713 views
Measuring distance preservation in dimensionality reduction
I am looking to compare the distance preserved during dimension reductions for several techniques. I have read some papers on similar topics here and here.
For example, I would like to use the ...
