Questions tagged [distance]

For question regarding distance between distributions or variables, such as Euclidean distance between points in n-space.

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

Metric for correlation of two paired lists of numbers

I have a program which produces an image, and I use a metric to understand how accurate that image is. I choose five cases (A, B, C, D, E), and make a list of the accuracy metric for each case: ...
user avatar
0 votes
0 answers
6 views

How to use distance matrix from csv in Orange for network explorer *without* using distance tool?

It seems the only way to create a network from distances is to use the distances tool, so I have to connect my distance matrix in csv format to the distances tool in order to create a network, which ...
user avatar
0 votes
0 answers
11 views

How to calculate distance for symmetric binary and nomianl variables?

In the existing function dist(), the only method for nominal variable is 'binary', and it's for asymmetric binary. However, I ...
user avatar
0 votes
0 answers
6 views

Search one 2D distribution for point cluster most similar to another 2D distribution

Given a hand drawn constellation (2d distribution of points) and a map of all stars, how would you find the actual star distribution most similar to the drawn distribution? If it's helpful, suppose we ...
user avatar
  • 184
0 votes
0 answers
15 views

matrix profile distance measure characterization

If there are various types of distances measures for time series, such as Euclidean, DTW, and shape-based ones, how can we characterize the matrix profile distance measure? Profiling one?
user avatar
2 votes
0 answers
54 views

Cosine-like alternative to Mahalanobis distance

I would like to have a distance measure that takes into account how spread are vectors in a dataset, to weight the absolute distance from one point to another. The Mahalanobis distance does exactly ...
user avatar
  • 21
1 vote
1 answer
97 views

Distance Metric between 2 lists of sets

I have 2 list of of sets and I want to calculate a distance. ...
user avatar
  • 143
1 vote
2 answers
31 views

Given daily sequence of events with only event ID labels (alphanum strings), what algorithms can be used to detect sequences that are outliers?

For example, the data might be something like this: ...
user avatar
  • 11
1 vote
0 answers
21 views

Does Sliced Wasserstein Distance work in higher than 2 dimensions?

I had thought that it only worked for 2D distributions. I am trying to implement a sliced Wasserstein autoencoder and I was wondering if my latent space can be larger than 2D.
user avatar
  • 35
-1 votes
2 answers
243 views

Vectorized String Distance

I am looking for a way to calculate the string distance between two Pandas dataframe columns in a vectorized way. I tried distance and textdistance libraries but they require to use df.apply which is ...
user avatar
  • 121
8 votes
3 answers
736 views

Levenshtein distance vs simple for loop

I have recently begun studying different data science principles, and have had a particular interest as of late in fuzzy matching. For preface, I'd like to include smarter fuzzy searching in a ...
user avatar
1 vote
2 answers
217 views

Can siamese model trained with euclidean distance as distance metric use cosine similarity during inference?

If I have 3 embeddings Anchor, Positive, Negative from a Siamese model trained with Euclidean distance as distance metric for triplet loss. During inference can ...
user avatar
0 votes
0 answers
37 views

What's an appropriate clustering quality estimate / metric for precomputed distance in HDBSCAN?

HBDSCAN supports estimation of clusters from precomputed distances. However, the python implementation of HDBSCAN (scikit-contrib) doesn't create minimum spanning trees in the absence of raw data when ...
user avatar
  • 1
2 votes
0 answers
12 views

Measuring the distance between data points based on mutual linkages

How to measure the distance between two data points (or: nodes?) based on their mutual share of linkages? I don't know the technical term for that, so here is a fictitious example from scientific ...
user avatar
  • 123
0 votes
0 answers
42 views

Hamming distance between column-wise of two matrices

Is there any MATLAB trick to perform hamming distance between columns of two different binary matrices(-1 & 1): A(64 by100) and B(64 by 80), and report the minimum distances? "A" is a ...
user avatar
1 vote
0 answers
18 views

Algorithm/method for grouping items based on their relative distance

I'm looking for a method to classify a set of items based on their relative distance. For example assume we have 4 cities and we know their relative distance: city1 city2 city3 city4 0 2.1 2.2 3.4 ...
user avatar
1 vote
0 answers
11 views

Similarity between binary vector with hierarchal structure

I have dataset of binary vectors, where each vector composed from several small vector coming from a different parent category. Each of those categories has a different size e.g. ...
user avatar
  • 111
1 vote
0 answers
22 views

Comparing the similarity structure of 2 distance matrices (computed from sentence embedding)

I apologize if this question lacks clarity, my mathematical background on the topic is limited and was hoping to find some guidance. I would like to compare 2 distance matrices that contain pair-wise ...
user avatar
  • 11
0 votes
0 answers
18 views

Generating unique points with an auto-encoder

I have been working on some research using a type of auto-encoder to generate new points with specific desirable properties. I trained my network and successfully generated some points, but when I ...
user avatar
1 vote
0 answers
199 views

Best way to find nearest neighbor distance for large datasets

I am a grad student doing research using generative machine learning with pytorch, and I have generated a set of points. I would like to check how similar these new points are to the points I used in ...
user avatar
0 votes
0 answers
89 views

K-NN algorithm with maximum distance to be considered a neighbor

Is there a variant of the k-NN algorithm where the label returned is: the average of values of the k nearest neighbors that are closer than a given threshold to the query data point? no value if ...
user avatar
0 votes
0 answers
107 views

How to calculate distance between points (with UTM coordinates) in Rstudio?

I have points on a QGIS map, and want to determine the distance between each of the points in Rstudio. Each Unique ID is a tree. The coordinates are UTM coordinates (x = East, y = North) My dataset ...
user avatar
0 votes
0 answers
10 views

Inner working behind combining two distance function as one function for similarity measure

I am comparing two images and for this I am testing various similarity function. For my case, Euclidean works much better than cosine(20% difference). However, I tried to combine two distance function ...
user avatar
0 votes
0 answers
16 views

Statistical method to validate predicted outliers

I was trying to make a clustering-based unsupervised anomaly detection on a large high-dimensional dataset. Roughly saying the points not lied inside all the clusters are defined as anomalies or ...
user avatar
1 vote
2 answers
128 views

If we dont specify any distance in KNN model, how is n_neighbors parameter calculated?

If we don’t specify the distance, how is the n_neighbors calculated?
user avatar
0 votes
0 answers
155 views

L1 vs. L2 Robustness?

I am very new to ML so I apologize in advance if the answer to my question is very obvious. I am reading about performance measures and how the L1 norm is more robust than the L2 norm. In other words, ...
user avatar
1 vote
0 answers
314 views

Real distance between bounding box centers

Assume that I have a camera pointing in a specific direction. I know the Euclidean distance (Real world distance) of the camera to a fixed point, X (mm). Using ...
user avatar
1 vote
0 answers
18 views

Testing similarity scores?

I need to calculate similarity between different houses using a series of attributes/properties, and to do that I need to define a certain similarity or distance function. Is there a way to test the ...
user avatar
1 vote
1 answer
362 views

How to calculate the distance between two locations using Haversine Formula? [closed]

I have the columns of Latitude and Longitude of city like shown below : ...
user avatar
  • 219
0 votes
1 answer
24 views

Treat similar observations in a classification problem

I have a dataset of about 200k rows and I'm working on a classification problem. Grouping the dataset by a key variable, I noticed that some rows, with the same key value, have similar values in other ...
user avatar
0 votes
1 answer
72 views

Non-commutative distance formula

I am trying to find a distance formula or a method that can give the non-commutative distance between two points in a feature space. Suppose there are two movies represented in an R^n feature space. ...
user avatar
0 votes
0 answers
130 views

2D Z-score/Mahalanobis distance that includes a penalty for uncertainty

I have some 2D points and I want to assess their performance against the target point. When I was doing this in 1D, I took the Z-score Z = (x- mu)/sigma, but that ...
user avatar
1 vote
0 answers
57 views

Compare distance between embeddings in different dimensions

I am working on a problem with CNNs. After the convolutional layers, comes a "flatten". One could interpret that as a representation of the input image in some high-dimensional continuous ...
user avatar
  • 13
1 vote
1 answer
77 views

Why does an imbalanced data set badly effect distance measures like Mahalanobis?

I'm relatively new to data science and I am struggling to understand why the Mahalanobis distance (or any other distance measure) applied to an imbalanced data-set becomes inaccurate. I have a data ...
user avatar
  • 13
1 vote
1 answer
472 views

How to estimate real distance between two detected objects in an image?

You may think this is a duplicate, but my situation is different than previously asked questions. The only information I have is the width and height of the bounding boxes of detected people. The ...
user avatar
0 votes
1 answer
117 views

Siamese vs matching network for correct image category matching

I have to find the closest match between my image and bunch of already collected images of different classes in the folder. Whic meta-learning approach should I select. I am thinking about the Siamese ...
user avatar
0 votes
0 answers
51 views

Use Machine Learning/Neural Network + Distance Measurements to Find the Position of Devices (Localization)

I want to find the position of several devices using at least distance measurements. These measurements are done using a radio, and it might be that not all devices are in radio range (no distance ...
user avatar
  • 1
0 votes
1 answer
62 views

Correlation/distance between sparse vectors

I am looking for a metric for comparing gene count tables. These are long columns of data (a few millions genes by a few dozen samples), with all non-negative entries, about 90% of which are zeros. ...
user avatar
0 votes
0 answers
24 views

What's the best way to detect crowds?

I have a dictionary containing people and the distance between each pair in the following format: ...
user avatar
  • 125
2 votes
1 answer
46 views

What are the Most Dissimilar MNIST Digits?

Using whatever definition of dissimilarity over sets that you'd like, what are the most dissimilar two digits in MNIST? I was thinking that a reasonable approach to answering the question would be to ...
user avatar
7 votes
1 answer
3k views

Why is the cosine distance used to measure the similatiry between word embeddings?

While computing the similarity between the words, cosine similarity or distance is computed on word vectors. Why aren't other distance metrics such as Euclidean ...
user avatar
4 votes
1 answer
111 views

KNN Regression: Distance function and/or vector representation for datetime features

Context: Trying to forecast some sort of consumption value (e.g. water) using datetime features and exogenous variables (like temperature). Take some datetime features like week days (...
user avatar
3 votes
2 answers
3k views

Similarity Measure between two feature vectors

I have face identification system with following details: VGG16 model for feature extraction 512 dimensional feature vector (...
user avatar
  • 133
1 vote
0 answers
43 views

Pairwise 3D object correlation between 2 objects

I have a dataset which contains 3D CT scans from different patients along with the segmenation masks of a certain organ. The 3D scans have been drawn each day for a period of 30 days for each patient. ...
user avatar
0 votes
1 answer
13 views

How to measure the distance (in generalized sense) between geographical regions? [closed]

I need to construct a distance matrix for a few U.S. counties that are adjacent to one or another, and choosing the definition of distance is very tricky. The shortest path (i.e the minimum number of ...
user avatar
0 votes
1 answer
45 views

Clustering without information about identifier

I have a data-set with different products and binary value if it was sold in a store or not. It looks like: product_id store_1 store_2 store_3 store_4 store_5 store_6 A 1 0 0 1 0 1 B 1 1 0 0 1 0 ...
user avatar
  • 121
6 votes
2 answers
107 views

How do I test a difference between two proportions representing fatality rate for Covid 19 in Philippines and World (except Philippines)?

I'm trying to analyse if the fatality rate from my country (A third world country) vary significantly from the world's fatality rate. So I'd basically have two samples, labeled (Philippines) and (...
user avatar
3 votes
4 answers
291 views

Clustering algorithm which does not require to tell the number of clusters

I have a dataframe with 2 columns of numerical values. I want to apply a clustering algorithm to put all the entries into the same group, which have a relatively small distance to the other entries. ...
user avatar
3 votes
1 answer
29 views

Very basic question: what is an accepted term for "linear order distance"

In data science we have "Manhattan Distance" as a slang term for Level 1 Distance and "Euclidean Distance" as a slang term for Level 2 Distance. Is there an accepted term for linear distance in memory ...
user avatar
5 votes
3 answers
473 views

Cluster elements that appear in the same lists

Suppose I have a multitude of sets with (unordered) combinations of elements and I want to determine which elements tend to appear together. For example Given the following sets: ...
user avatar