Questions tagged [distance]

For question regarding distance between distributions or variables, such as Euclidean distance between points in n-space.

Filter by
Sorted by
Tagged with
0 votes
0 answers
15 views

Top hundred nearest neighbour

I have a dataframe with a column called pharmacy number and other columns corresponding to each pharmacy number, there many rows and each row corresponds to pharmacy number. I want to create a ...
Sanket Maiti's user avatar
0 votes
0 answers
10 views

question regarding scaling for k-means clustering

is it correct to say this? we need to scale every numeric variable because if we don't, a variable with a large range of variance will dominate. So, one way to judge a variable with a large range of ...
user392987's user avatar
0 votes
0 answers
42 views

Using "precomputed" distance matrices as input to scikit-learn clustering metrics

Is there any validity to using a distance matrix instead of the raw points with metrics such as davies_bouldin_score and ...
Chris Coffee's user avatar
0 votes
0 answers
13 views

Best distance metric and estadarization method for clustering with percentages data

I'm studying access patterns to a facility with clustering. My variables are percentages. For example, for each user, I have the percentage of access 'in time' versus late, or the percentage of using ...
Kaikus's user avatar
  • 101
0 votes
0 answers
11 views

Examples of distance "hyperparameters" used in clustering

From what I've seen in clustering, distance is taken as a hyper parameter (which is to be selected) when inferring the relationships/clusters between points. What are some examples of highly-cited ...
ABIM's user avatar
  • 123
0 votes
1 answer
65 views

Standard metric for distance between two clusters

Let $A=\{A_1,A_2,\cdots,A_m\}$ and $B=\{B_1,B_2,\cdots,B_n\}$ be two sets of points in $k$-dimensional Euclidean space. Each points $A_i$ or $B_i$ can be thought of as a feature vector of a data ...
govindah's user avatar
0 votes
1 answer
38 views

Depth Estimation Algorithms without Reference Image in Computer Vision for Webcam Captured Video Data of a Person

I am currently working on a computer vision project that involves analyzing video data of a person captured from a webcam. In this project, I need to compute the depth map or distance of a specific ...
thedumbkid's user avatar
1 vote
0 answers
30 views

What is the l2-norm of a scalar

What is the meaning of the l2-norm when dealing with scalar values? I'm assuming it would be the same thing as taking the absolute value. For context: I am trying to implement the clustering method ...
Droidenkiller's user avatar
0 votes
0 answers
15 views

How can a pullback dissimilarity on a nasty space be interpolated/approximated?

I have a map $\gamma : X \rightarrow Y$ that is expensive to compute. $X$ is a nasty, very non-Euclidean, not even manifold-like, space of variable-length and "structurally inhomogeneous" ...
Steve Huntsman's user avatar
0 votes
0 answers
38 views

Distance Metric for a dataset with embeddings and numerical columns

I'm trying to build an Approximate Nearest Neighbours model that can fetch similar records in a dataset, that are contextually similar. For example, for a record of job=Software Engineer and age=25, a ...
Augustine Samuel's user avatar
0 votes
1 answer
47 views

What is this "F" subscript symbol that shows up in this loss function?

i was reading this https://arxiv.org/pdf/2303.14535v1.pdf paper when i came across this: What is this F? Initially i assumed this was a standard L2 distance but i'm not so sure anymore.
Alessandro Polidori's user avatar
1 vote
0 answers
32 views

In WGAN paper, why does clipping weights approximate Lipschitz function?

In Wasserstein GAN, it's explained that maximizing a certain formula over a set of K-Lipschitz functions approximates the 1-Wasserstein distance and they model the functions as NNs. That much I ...
znb's user avatar
  • 11
1 vote
1 answer
32 views

Using human created small groups to identify entirely new small groups

I'm relatively new to data science, but an old hat at analytics. I'm looking for some direction on a project that I'm wanting to work on. I'm working with discrete objects, that when small changes ...
Ryan Fry's user avatar
1 vote
1 answer
205 views

In DBSCAN, can the distance between a Noise Point and Border Point be less than Epsilon?

In DBSCAN: A core point is a point which has at least "MinPts" points inside its Epsilon radius. A border point is a point inside the Epsilon radius of a core point, but it has a number of ...
SuperFluo's user avatar
0 votes
0 answers
335 views

How to do DBSCAN clustering with mixed variables (numerical features and binary/ordinal variables)?

I have a question written at the end of the post which refers to the "Distances" paragraph. The other first two paragraphs give additional info. Context I'm working on a project where I have ...
SuperFluo's user avatar
0 votes
0 answers
17 views

Critique my algorithm for measuring similar/difference of groups using multiple variables

So I've been trying to solve a problem of quantitatively measuring the similarity/difference between groups in my dataset. I am not trying to cluster data to create groups, because the groups are ...
pubb's user avatar
  • 1
1 vote
1 answer
119 views

Nearest Neighbor Recommendation System w/ categorical variables

I would like to build a recommendation system: no ratings are available at the time of recommendation, therefore only a purely context-based recommendation system is needed as input features answers ...
alexryder's user avatar
3 votes
1 answer
90 views

What justifies feature scaling?

Although I can understand the significance of feature scaling in some cases (e.g. when gradient descent is involved), I don't feel I understand the necessity of this process in general. But there a ...
ado sar's user avatar
  • 181
0 votes
0 answers
42 views

matrix profile distance measure characterization

If there are various types of distances measures for time series, such as Euclidean, DTW, and shape-based ones, how can we characterize the matrix profile distance measure? Profiling one?
user18602524's user avatar
2 votes
0 answers
332 views

Cosine-like alternative to Mahalanobis distance

I would like to have a distance measure that takes into account how spread are vectors in a dataset, to weight the absolute distance from one point to another. The Mahalanobis distance does exactly ...
a_gdevr's user avatar
  • 21
1 vote
1 answer
147 views

Distance Metric between 2 lists of sets

I have 2 list of of sets and I want to calculate a distance. ...
pettinato's user avatar
  • 143
1 vote
2 answers
56 views

Given daily sequence of events with only event ID labels (alphanum strings), what algorithms can be used to detect sequences that are outliers?

For example, the data might be something like this: ...
demoman's user avatar
  • 11
1 vote
0 answers
52 views

Does Sliced Wasserstein Distance work in higher than 2 dimensions?

I had thought that it only worked for 2D distributions. I am trying to implement a sliced Wasserstein autoencoder and I was wondering if my latent space can be larger than 2D.
nighthawk's user avatar
0 votes
2 answers
1k views

Vectorized String Distance

I am looking for a way to calculate the string distance between two Pandas dataframe columns in a vectorized way. I tried distance and textdistance libraries but they require to use df.apply which is ...
Anatole's user avatar
  • 181
8 votes
3 answers
967 views

Levenshtein distance vs simple for loop

I have recently begun studying different data science principles, and have had a particular interest as of late in fuzzy matching. For preface, I'd like to include smarter fuzzy searching in a ...
Jadon Steinmetz's user avatar
1 vote
2 answers
858 views

Can siamese model trained with euclidean distance as distance metric use cosine similarity during inference?

If I have 3 embeddings Anchor, Positive, Negative from a Siamese model trained with Euclidean distance as distance metric for triplet loss. During inference can ...
B200011011's user avatar
2 votes
0 answers
16 views

Measuring the distance between data points based on mutual linkages

How to measure the distance between two data points (or: nodes?) based on their mutual share of linkages? I don't know the technical term for that, so here is a fictitious example from scientific ...
anpami's user avatar
  • 123
1 vote
0 answers
21 views

Algorithm/method for grouping items based on their relative distance

I'm looking for a method to classify a set of items based on their relative distance. For example assume we have 4 cities and we know their relative distance: city1 city2 city3 city4 0 2.1 2.2 3.4 ...
Mehdi Zare's user avatar
1 vote
0 answers
14 views

Similarity between binary vector with hierarchal structure

I have dataset of binary vectors, where each vector composed from several small vector coming from a different parent category. Each of those categories has a different size e.g. ...
Amit be's user avatar
  • 111
1 vote
0 answers
38 views

Comparing the similarity structure of 2 distance matrices (computed from sentence embedding)

I apologize if this question lacks clarity, my mathematical background on the topic is limited and was hoping to find some guidance. I would like to compare 2 distance matrices that contain pair-wise ...
keun's user avatar
  • 11
0 votes
0 answers
24 views

Generating unique points with an auto-encoder

I have been working on some research using a type of auto-encoder to generate new points with specific desirable properties. I trained my network and successfully generated some points, but when I ...
Amateur Coding Bird's user avatar
1 vote
0 answers
661 views

Best way to find nearest neighbor distance for large datasets

I am a grad student doing research using generative machine learning with pytorch, and I have generated a set of points. I would like to check how similar these new points are to the points I used in ...
Amateur Coding Bird's user avatar
1 vote
0 answers
323 views

K-NN algorithm with maximum distance to be considered a neighbor

Is there a variant of the k-NN algorithm where the label returned is: the average of values of the k nearest neighbors that are closer than a given threshold to the query data point? no value if ...
maxime langevin's user avatar
0 votes
0 answers
174 views

How to calculate distance between points (with UTM coordinates) in Rstudio?

I have points on a QGIS map, and want to determine the distance between each of the points in Rstudio. Each Unique ID is a tree. The coordinates are UTM coordinates (x = East, y = North) My dataset ...
Burton Guster's user avatar
1 vote
2 answers
170 views

If we dont specify any distance in KNN model, how is n_neighbors parameter calculated?

If we don’t specify the distance, how is the n_neighbors calculated?
Karthik Ganesh's user avatar
0 votes
0 answers
389 views

L1 vs. L2 Robustness?

I am very new to ML so I apologize in advance if the answer to my question is very obvious. I am reading about performance measures and how the L1 norm is more robust than the L2 norm. In other words, ...
Jonathan Duran's user avatar
1 vote
0 answers
712 views

Real distance between bounding box centers

Assume that I have a camera pointing in a specific direction. I know the Euclidean distance (Real world distance) of the camera to a fixed point, X (mm). Using ...
Serge de Gosson de Varennes's user avatar
1 vote
0 answers
27 views

Testing similarity scores?

I need to calculate similarity between different houses using a series of attributes/properties, and to do that I need to define a certain similarity or distance function. Is there a way to test the ...
Max van der Werf's user avatar
1 vote
1 answer
2k views

How to calculate the distance between two locations using Haversine Formula? [closed]

I have the columns of Latitude and Longitude of city like shown below : ...
Hamza's user avatar
  • 229
0 votes
1 answer
50 views

Treat similar observations in a classification problem

I have a dataset of about 200k rows and I'm working on a classification problem. Grouping the dataset by a key variable, I noticed that some rows, with the same key value, have similar values in other ...
Roberto Buzzini's user avatar
0 votes
1 answer
163 views

Non-commutative distance formula

I am trying to find a distance formula or a method that can give the non-commutative distance between two points in a feature space. Suppose there are two movies represented in an R^n feature space. ...
Himanshu's user avatar
0 votes
0 answers
167 views

2D Z-score/Mahalanobis distance that includes a penalty for uncertainty

I have some 2D points and I want to assess their performance against the target point. When I was doing this in 1D, I took the Z-score Z = (x- mu)/sigma, but that ...
Lizardinablizzard's user avatar
1 vote
0 answers
157 views

Compare distance between embeddings in different dimensions

I am working on a problem with CNNs. After the convolutional layers, comes a "flatten". One could interpret that as a representation of the input image in some high-dimensional continuous ...
nico_so's user avatar
  • 13
1 vote
1 answer
111 views

Why does an imbalanced data set badly effect distance measures like Mahalanobis?

I'm relatively new to data science and I am struggling to understand why the Mahalanobis distance (or any other distance measure) applied to an imbalanced data-set becomes inaccurate. I have a data ...
Jimonty's user avatar
  • 21
1 vote
1 answer
1k views

How to estimate real distance between two detected objects in an image?

You may think this is a duplicate, but my situation is different than previously asked questions. The only information I have is the width and height of the bounding boxes of detected people. The ...
Enzo1912's user avatar
0 votes
1 answer
228 views

Siamese vs matching network for correct image category matching

I have to find the closest match between my image and bunch of already collected images of different classes in the folder. Whic meta-learning approach should I select. I am thinking about the Siamese ...
Rambo_john's user avatar
0 votes
0 answers
74 views

Use Machine Learning/Neural Network + Distance Measurements to Find the Position of Devices (Localization)

I want to find the position of several devices using at least distance measurements. These measurements are done using a radio, and it might be that not all devices are in radio range (no distance ...
EVRR's user avatar
  • 1
0 votes
1 answer
153 views

Correlation/distance between sparse vectors

I am looking for a metric for comparing gene count tables. These are long columns of data (a few millions genes by a few dozen samples), with all non-negative entries, about 90% of which are zeros. ...
Roger V.'s user avatar
  • 101
0 votes
0 answers
25 views

What's the best way to detect crowds?

I have a dictionary containing people and the distance between each pair in the following format: ...
Maf's user avatar
  • 125
2 votes
1 answer
189 views

What are the Most Dissimilar MNIST Digits?

Using whatever definition of dissimilarity over sets that you'd like, what are the most dissimilar two digits in MNIST? I was thinking that a reasonable approach to answering the question would be to ...
JoeTheShmoe's user avatar